Jul 15 18:49:00.235334 ip-10-0-137-11 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 18:49:00.235345 ip-10-0-137-11 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 18:49:00.235351 ip-10-0-137-11 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 18:49:00.235697 ip-10-0-137-11 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 18:49:10.275498 ip-10-0-137-11 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 15 18:49:10.275513 ip-10-0-137-11 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f571a130bebd4241973db3ba885b9b45 -- Jul 15 18:50:48.199962 ip-10-0-137-11 systemd[1]: Starting Kubernetes Kubelet... Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.656655 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663075 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663086 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663089 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663093 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663096 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663099 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663102 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:50:48.737947 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663105 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663107 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663110 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663114 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663116 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663119 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663121 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663124 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663126 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663129 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663131 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663134 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663136 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663139 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663141 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663144 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663146 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663148 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663151 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663153 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:50:48.740360 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663164 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:50:48.732448 ip-10-0-137-11 systemd[1]: Started Kubernetes Kubelet. Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663167 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663170 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663172 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663175 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663178 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663180 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663182 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663185 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663188 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663190 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663192 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663195 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663199 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663205 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663208 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663210 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663213 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663215 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663218 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:50:48.753972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663221 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663223 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663226 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663230 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663234 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663237 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663241 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663244 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663247 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663250 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663253 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663255 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663258 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663260 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663263 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663266 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663268 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663271 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663273 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:50:48.757579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663276 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663279 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663281 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663284 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663287 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663289 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663292 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663295 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663297 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663299 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663302 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663305 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663307 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663310 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663312 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663315 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663317 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663320 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663323 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:50:48.759818 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663326 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663329 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663331 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663333 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663336 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663338 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663341 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663343 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663346 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663349 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663351 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663354 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663356 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.663359 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664009 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664016 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664020 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664023 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664025 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664028 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:50:48.762011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664031 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664033 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664036 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664039 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664041 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664044 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664046 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664050 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664052 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664055 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664057 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664060 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664062 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664065 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664067 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664069 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664071 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664074 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664076 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664079 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:50:48.763835 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664082 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664084 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664087 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664089 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664093 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664096 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664099 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664101 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664104 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664106 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664109 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664111 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664113 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664116 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664119 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664121 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664124 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664126 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664129 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664132 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:50:48.767131 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664135 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664137 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664140 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664142 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664145 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664147 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664150 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664154 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664158 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664161 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664165 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664168 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664172 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664175 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664178 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664180 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664184 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664186 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664189 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:50:48.782943 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664192 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664194 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664198 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664201 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664203 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664206 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664209 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664212 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664214 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664219 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664223 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664226 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664228 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664231 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664235 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664237 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664240 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664242 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664245 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:50:48.936562 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664247 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664250 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664252 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664254 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664257 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664259 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664262 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664265 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664267 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664270 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664273 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664275 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664278 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664280 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664283 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664627 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664644 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664647 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664650 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664652 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:50:48.954405 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664656 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664658 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664661 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664681 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664684 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664687 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664689 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664693 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664695 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664698 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664701 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664703 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664726 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664729 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664734 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664739 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664743 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664746 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664749 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664752 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:50:48.956536 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664755 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664757 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664760 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664762 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664765 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664768 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664771 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664775 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664778 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664781 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664784 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664786 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664789 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664792 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664796 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664800 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664803 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664806 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664808 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:50:48.958460 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664811 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664814 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664817 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664819 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664822 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664824 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664827 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664831 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664834 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664836 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664839 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664841 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664843 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664846 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664849 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664851 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664853 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664856 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664859 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:50:48.961379 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664861 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664864 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664866 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664869 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664871 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664874 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664876 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664879 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664883 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664885 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664888 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664891 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664893 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664896 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664898 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664910 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664913 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664915 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664917 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664920 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:50:48.962494 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664924 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664927 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664929 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664932 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664934 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664938 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664941 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664944 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664947 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664950 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664954 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664956 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664959 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664961 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664964 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.664966 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665029 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665036 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665041 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665046 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665051 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 18:50:48.963586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665055 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665061 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665066 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665071 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665075 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665080 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665084 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665101 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665105 2574 flags.go:64] FLAG: --cgroup-root="" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665108 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665111 2574 flags.go:64] FLAG: --client-ca-file="" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665115 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665118 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665127 2574 flags.go:64] FLAG: --cluster-domain="" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665130 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665133 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665136 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665139 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665143 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665145 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665148 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665152 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665155 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665157 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 18:50:48.964387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665160 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665163 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665166 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665170 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665173 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665176 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665179 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665184 2574 flags.go:64] FLAG: --enable-server="true" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665187 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665193 2574 flags.go:64] FLAG: --event-burst="100" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665196 2574 flags.go:64] FLAG: --event-qps="50" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665199 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665202 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665205 2574 flags.go:64] FLAG: --eviction-hard="" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665209 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665212 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665220 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665224 2574 flags.go:64] FLAG: --eviction-soft="" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665226 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665231 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665234 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665236 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665239 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665242 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665245 2574 flags.go:64] FLAG: --feature-gates="" Jul 15 18:50:48.965526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665248 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665251 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665254 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665257 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665260 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665263 2574 flags.go:64] FLAG: --help="false" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665266 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-137-11.ec2.internal" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665281 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665284 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665287 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665290 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665294 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665296 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665300 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665304 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665307 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665309 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665312 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665315 2574 flags.go:64] FLAG: --kube-reserved="" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665318 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665321 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665324 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665327 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665329 2574 flags.go:64] FLAG: --lock-file="" Jul 15 18:50:48.966626 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665338 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665341 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665345 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665354 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665357 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665360 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665363 2574 flags.go:64] FLAG: --logging-format="text" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665366 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665369 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665371 2574 flags.go:64] FLAG: --manifest-url="" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665374 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665378 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665381 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665385 2574 flags.go:64] FLAG: --max-pods="110" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665388 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665390 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665393 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665396 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665399 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665402 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665405 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665412 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665416 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665419 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 18:50:48.967704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665422 2574 flags.go:64] FLAG: --pod-cidr="" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665424 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665430 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665433 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665436 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665439 2574 flags.go:64] FLAG: --port="10250" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665442 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665444 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00eff207b2e914467" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665448 2574 flags.go:64] FLAG: --qos-reserved="" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665456 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665461 2574 flags.go:64] FLAG: --register-node="true" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665464 2574 flags.go:64] FLAG: --register-with-taints="" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665467 2574 flags.go:64] FLAG: --registry-burst="10" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665470 2574 flags.go:64] FLAG: --registry-qps="5" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665473 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665475 2574 flags.go:64] FLAG: --reserved-memory="" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665479 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665482 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665485 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665488 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665491 2574 flags.go:64] FLAG: --runonce="false" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665493 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665496 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665499 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665502 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 18:50:48.972160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665505 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665508 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665511 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665514 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665517 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665520 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665524 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665527 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665530 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665532 2574 flags.go:64] FLAG: --system-cgroups="" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665535 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665540 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665543 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665546 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665553 2574 flags.go:64] FLAG: --tls-min-version="" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665555 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665558 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665569 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665572 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665575 2574 flags.go:64] FLAG: --v="2" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665579 2574 flags.go:64] FLAG: --version="false" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665582 2574 flags.go:64] FLAG: --vmodule="" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665586 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.665589 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665743 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:50:48.973485 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665748 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665751 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665754 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665757 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665759 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665762 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665765 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665768 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665770 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665773 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665776 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665778 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665780 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665785 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665787 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665792 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665796 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665799 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665802 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665805 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:50:48.981086 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665808 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665810 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665813 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665815 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665820 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665828 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665831 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665834 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665836 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665839 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665841 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665844 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665846 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665849 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665864 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665867 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665869 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665872 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665874 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665876 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:50:48.981971 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665879 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665881 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665884 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665886 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665890 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665895 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665898 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665900 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665903 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665905 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665908 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665910 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665913 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665915 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665918 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665920 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665925 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665927 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665931 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665934 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:50:48.983001 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665936 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665938 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665942 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665945 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665948 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665951 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665954 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665957 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665959 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665962 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665965 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665967 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665970 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665972 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665975 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665977 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665980 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665984 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665986 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:50:48.983862 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665989 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665991 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665994 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665997 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.665999 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666002 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666004 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666006 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666009 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666013 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666015 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666018 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666021 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666023 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666026 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666028 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666030 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666033 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:50:48.984743 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.666035 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.666811 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.673634 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.673648 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.673669 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.673675 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673755 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673761 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673765 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673769 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673773 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673777 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673780 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673783 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673787 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:50:48.985579 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673790 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673794 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673797 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673800 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673804 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673807 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673810 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673813 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673817 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673820 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673823 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673826 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673830 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673833 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673839 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673842 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673860 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673866 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673872 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:50:48.986239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673877 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673881 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673885 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673889 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673892 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673895 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673899 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673902 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673905 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673909 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673912 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673915 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673918 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673922 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673926 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673929 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673933 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673936 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673939 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673942 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:50:48.987073 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673945 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673948 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673951 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673954 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673956 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673959 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673961 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673966 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673968 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673971 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673973 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673977 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673979 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673982 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673988 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673991 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673994 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673996 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.673999 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674001 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:50:48.987915 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674004 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674007 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674009 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674011 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674014 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674017 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674019 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674022 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674024 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674026 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674029 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674032 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674034 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674037 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674039 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674042 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674044 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674047 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674049 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:50:48.989011 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674053 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674055 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674058 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674060 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674065 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674069 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674072 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674076 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674080 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674084 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674087 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674091 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.674098 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674257 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:50:48.989876 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674266 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674270 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674273 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674277 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674280 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674283 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674285 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674288 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674290 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674293 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674296 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674298 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674301 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674303 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674306 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674309 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674311 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674314 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674316 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:50:48.990716 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674321 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674324 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674326 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674330 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674333 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674336 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674338 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674341 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674344 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674348 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674351 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674354 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674357 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674360 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674363 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674366 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674369 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674372 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674374 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674377 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:50:48.991555 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674380 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674382 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674385 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674387 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674389 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674392 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674394 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674397 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674399 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674402 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674404 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674407 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674411 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674413 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674416 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674419 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674422 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674425 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674428 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674430 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:50:48.992403 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674433 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674436 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674440 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674444 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674447 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674449 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674452 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674454 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674457 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674460 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674462 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674465 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674468 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674470 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674472 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674475 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674477 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674480 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674483 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674485 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:50:48.993309 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674487 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674490 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674492 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674495 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674499 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674501 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674504 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674507 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674510 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674512 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674515 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674517 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674521 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674524 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674528 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674531 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674533 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674536 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:50:48.994199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:48.674538 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.674543 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.674641 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.677026 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.678788 2574 server.go:1019] "Starting client certificate rotation" Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.678885 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.678922 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.683334 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.685169 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.695383 2574 log.go:25] "Validated CRI v1 runtime API" Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.698190 2574 log.go:25] "Validated CRI v1 image API" Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.699609 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 18:50:48.995064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.702265 2574 fs.go:135] Filesystem UUIDs: map[332ac238-c95b-4e0d-96c6-b581421e6fb6:/dev/nvme0n1p4 5fe88c69-ee89-42e6-83af-1d6c9ae2245e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 15 18:50:49.219092 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.702278 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 18:50:49.220025 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.707346 2574 manager.go:216] Machine: {Timestamp:2026-07-15 18:50:48.705868869 +0000 UTC m=+0.372656937 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3157252 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2def048702e5e8dcc023af2efdca91 SystemUUID:ec2def04-8702-e5e8-dcc0-23af2efdca91 BootID:f571a130-bebd-4241-973d-b3ba885b9b45 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6b:ac:e8:0a:61 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6b:ac:e8:0a:61 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:01:bc:27:9a:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 18:50:49.220025 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.707446 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 18:50:49.220025 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.707517 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 18:50:49.220025 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.711081 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 18:50:49.220025 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.711109 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-11.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 18:50:49.220025 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.712004 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 18:50:49.220025 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.712013 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 18:50:49.220025 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.712020 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.712861 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.714686 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.719418 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.719535 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.719557 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.719560 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.719567 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.719660 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.719671 2574 kubelet.go:398] "Adding apiserver pod source" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.719681 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.720024 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.720047 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.722539 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.726465 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.726492 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.726643 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.726655 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.726662 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.726667 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.726677 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.726682 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.727279 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.727286 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.727292 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.727298 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.727306 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.727314 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.728213 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.728222 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.731570 2574 server.go:1268] "Started kubelet" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.731657 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.731647 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.731841 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.732318 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-11.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.732945 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.732946 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.733284 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.733766 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 15 18:50:49.220655 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.737369 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.739350 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-11.ec2.internal.18c28b31e0b1c726 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-11.ec2.internal,UID:ip-10-0-137-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-11.ec2.internal,},FirstTimestamp:2026-07-15 18:50:48.731543334 +0000 UTC m=+0.398331384,LastTimestamp:2026-07-15 18:50:48.731543334 +0000 UTC m=+0.398331384,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-11.ec2.internal,}" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.741687 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.741696 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.741770 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.742461 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.742464 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.742480 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.742559 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.742566 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.743953 2574 factory.go:55] Registering systemd factory Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.743965 2574 factory.go:223] Registration of the systemd container factory successfully Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.744154 2574 factory.go:153] Registering CRI-O factory Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.744171 2574 factory.go:223] Registration of the crio container factory successfully Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.744230 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.744254 2574 factory.go:103] Registering Raw factory Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.744269 2574 manager.go:1196] Started watching for new ooms in manager Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.744835 2574 manager.go:349] Starting recovery of all containers Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.745923 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.745968 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.758397 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vdjvq" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.758945 2574 manager.go:354] Recovery completed Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.763953 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.765569 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vdjvq" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.765985 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.766011 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.766030 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.766600 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.766609 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.766625 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.766973 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.767658 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:50:49.223174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.767687 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.767696 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.768280 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-11.ec2.internal.18c28b31e2bf86c5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-11.ec2.internal,UID:ip-10-0-137-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-11.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-11.ec2.internal,},FirstTimestamp:2026-07-15 18:50:48.765998789 +0000 UTC m=+0.432786841,LastTimestamp:2026-07-15 18:50:48.765998789 +0000 UTC m=+0.432786841,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-11.ec2.internal,}" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.769263 2574 policy_none.go:49] "None policy: Start" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.769278 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.769288 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.771251 2574 policy_none.go:47] "Start" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.814943 2574 manager.go:337] "Starting Device Plugin manager" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.814970 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.814979 2574 server.go:87] "Starting device plugin registration server" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.815224 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.815238 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.815370 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.815441 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.815448 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.815867 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.815911 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.879559 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.880785 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.880832 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.880857 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.880895 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.883298 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.916291 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.917206 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.917238 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.917249 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.917272 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.937517 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:48.937538 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-11.ec2.internal\": node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.998629 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-11.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal"] Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.998684 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.999530 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:50:49.224039 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.999554 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:48.999568 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.001813 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.001897 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.001920 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.002454 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.002483 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.002493 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.003021 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.003044 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.003056 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:49.003404 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.005217 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.005239 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.005829 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.005856 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.005882 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:49.040215 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-11.ec2.internal\" not found" node="ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.044357 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6eba61179e763a6a602c3d40734955c6-config\") pod \"kube-apiserver-proxy-ip-10-0-137-11.ec2.internal\" (UID: \"6eba61179e763a6a602c3d40734955c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.044391 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e798cb305b1aaccf6fbad4a0b532fc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal\" (UID: \"4e798cb305b1aaccf6fbad4a0b532fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.044415 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e798cb305b1aaccf6fbad4a0b532fc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal\" (UID: \"4e798cb305b1aaccf6fbad4a0b532fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:49.044814 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-11.ec2.internal\" not found" node="ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:49.103698 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.145136 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6eba61179e763a6a602c3d40734955c6-config\") pod \"kube-apiserver-proxy-ip-10-0-137-11.ec2.internal\" (UID: \"6eba61179e763a6a602c3d40734955c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.225660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.145159 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e798cb305b1aaccf6fbad4a0b532fc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal\" (UID: \"4e798cb305b1aaccf6fbad4a0b532fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.411024 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.145176 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e798cb305b1aaccf6fbad4a0b532fc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal\" (UID: \"4e798cb305b1aaccf6fbad4a0b532fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.411024 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.145229 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6eba61179e763a6a602c3d40734955c6-config\") pod \"kube-apiserver-proxy-ip-10-0-137-11.ec2.internal\" (UID: \"6eba61179e763a6a602c3d40734955c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.411024 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.145288 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e798cb305b1aaccf6fbad4a0b532fc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal\" (UID: \"4e798cb305b1aaccf6fbad4a0b532fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.411024 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.145357 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e798cb305b1aaccf6fbad4a0b532fc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal\" (UID: \"4e798cb305b1aaccf6fbad4a0b532fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.411024 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:49.204132 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:49.411024 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:49.304227 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:49.411024 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.342903 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.411024 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.347897 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" Jul 15 18:50:49.411024 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:49.404944 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:49.576996 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:49.505464 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:49.576996 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:49.568582 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:50:51.116839 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:50.924937 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:51.116839 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:50.997391 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 18:50:51.116839 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:50.997525 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 18:50:51.116839 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:50.997525 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 18:50:51.116839 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:51.025678 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:51.116839 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:51.089465 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 18:45:50 +0000 UTC" deadline="2028-04-16 22:18:08.936864316 +0000 UTC" Jul 15 18:50:51.116839 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:51.089486 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15387h27m17.847381558s" Jul 15 18:50:51.296741 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:51.126571 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:51.296741 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:51.227197 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-11.ec2.internal\" not found" Jul 15 18:50:51.296741 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:51.248740 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:50:51.296741 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:51.261218 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-11.ec2.internal" Jul 15 18:50:51.296741 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:51.272538 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 18:50:51.296741 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:51.273609 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" Jul 15 18:50:51.296741 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:51.283937 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 18:50:51.490096 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:51.369513 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e798cb305b1aaccf6fbad4a0b532fc3.slice/crio-7e3ce197a9585ce8022609cc8c0528755b264db9697502e2953e9c94fb091280 WatchSource:0}: Error finding container 7e3ce197a9585ce8022609cc8c0528755b264db9697502e2953e9c94fb091280: Status 404 returned error can't find the container with id 7e3ce197a9585ce8022609cc8c0528755b264db9697502e2953e9c94fb091280 Jul 15 18:50:51.490096 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:51.379250 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 18:50:51.490096 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:51.381851 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eba61179e763a6a602c3d40734955c6.slice/crio-55f221590b49170244a873b2544666d1911f2f090443105ab9bc74839f4295c6 WatchSource:0}: Error finding container 55f221590b49170244a873b2544666d1911f2f090443105ab9bc74839f4295c6: Status 404 returned error can't find the container with id 55f221590b49170244a873b2544666d1911f2f090443105ab9bc74839f4295c6 Jul 15 18:50:51.677661 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:51.531286 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:50:52.096192 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.040334 2574 apiserver.go:52] "Watching apiserver" Jul 15 18:50:52.096192 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.047271 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 18:50:52.096192 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.047669 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-qc4kh","openshift-network-operator/iptables-alerter-nhhtr","openshift-ovn-kubernetes/ovnkube-node-cjxwn","kube-system/global-pull-secret-syncer-z65fv","kube-system/konnectivity-agent-k5k4q","kube-system/kube-apiserver-proxy-ip-10-0-137-11.ec2.internal","openshift-dns/node-resolver-nsvqx","openshift-multus/multus-additional-cni-plugins-xcdnw","openshift-multus/network-metrics-daemon-9669b","openshift-network-diagnostics/network-check-target-mw4n7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd","openshift-cluster-node-tuning-operator/tuned-m8xkn","openshift-image-registry/node-ca-2mlqx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal"] Jul 15 18:50:52.096192 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.050223 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nsvqx" Jul 15 18:50:52.096192 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.052538 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2mlqx" Jul 15 18:50:52.096192 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.053378 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 18:50:52.096192 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.053387 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-44l4b\"" Jul 15 18:50:52.096192 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.054710 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.096192 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.055609 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 18:50:52.096192 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.095860 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.096412 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.096489 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.096558 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.096703 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.096722 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8rsz6\"" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.096775 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.096857 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.096868 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.096955 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2rj6s\"" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.097002 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.097016 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:50:52.097093 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.097102 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 18:50:52.097725 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.097123 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:52.097725 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.097193 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:50:52.097725 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.097277 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.097725 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.097417 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:52.097725 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.097469 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:50:52.099664 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.099259 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.102027 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nhhtr" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.102268 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.102509 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.102564 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.103473 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.103612 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.103991 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.104463 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.104744 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.105370 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ck2bd\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.107837 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p6p59\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.108910 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.111662 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.112103 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.112342 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.112442 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pckkw\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.112529 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.112629 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.113927 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5vgk2\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.116174 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.116563 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.117424 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k9fpq\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.117426 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.117471 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.118216 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.118490 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6nq49\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.118705 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.122167 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.138508 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nfgnq" Jul 15 18:50:52.188892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.148272 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nfgnq" Jul 15 18:50:52.190706 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.163075 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.197833 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-node-log\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.197866 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae918cfe-ebcc-4398-b50c-dacff69aa164-dbus\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.197891 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-lib-modules\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.197926 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt4wc\" (UniqueName: \"kubernetes.io/projected/72394437-d6bc-4577-8e92-ce60aec0f34a-kube-api-access-xt4wc\") pod \"node-resolver-nsvqx\" (UID: \"72394437-d6bc-4577-8e92-ce60aec0f34a\") " pod="openshift-dns/node-resolver-nsvqx" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.197949 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-kubelet\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.197984 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198019 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-device-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198045 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-cnibin\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198068 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-run-k8s-cni-cncf-io\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198088 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72394437-d6bc-4577-8e92-ce60aec0f34a-tmp-dir\") pod \"node-resolver-nsvqx\" (UID: \"72394437-d6bc-4577-8e92-ce60aec0f34a\") " pod="openshift-dns/node-resolver-nsvqx" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198123 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/076528ce-8f31-4354-8412-26284d944989-host-slash\") pod \"iptables-alerter-nhhtr\" (UID: \"076528ce-8f31-4354-8412-26284d944989\") " pod="openshift-network-operator/iptables-alerter-nhhtr" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198136 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppfpx\" (UniqueName: \"kubernetes.io/projected/f95dfaa6-6b45-4541-8827-af5e7ab0f492-kube-api-access-ppfpx\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.198363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198150 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-var-lib-cni-multus\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198215 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-slash\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198246 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-ovnkube-script-lib\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198285 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-sys-fs\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198307 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-sys\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198322 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-os-release\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198336 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-socket-dir-parent\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198353 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-run-ovn-kubernetes\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198367 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlczt\" (UniqueName: \"kubernetes.io/projected/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-kube-api-access-nlczt\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198380 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w45c6\" (UniqueName: \"kubernetes.io/projected/076528ce-8f31-4354-8412-26284d944989-kube-api-access-w45c6\") pod \"iptables-alerter-nhhtr\" (UID: \"076528ce-8f31-4354-8412-26284d944989\") " pod="openshift-network-operator/iptables-alerter-nhhtr" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198396 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-socket-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198418 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198438 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-cni-dir\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.198856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198452 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-daemon-config\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198464 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198477 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0673f240-f3b2-45cc-bfe2-fe7d49b5cd04-host\") pod \"node-ca-2mlqx\" (UID: \"0673f240-f3b2-45cc-bfe2-fe7d49b5cd04\") " pod="openshift-image-registry/node-ca-2mlqx" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198491 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qv55\" (UniqueName: \"kubernetes.io/projected/0673f240-f3b2-45cc-bfe2-fe7d49b5cd04-kube-api-access-7qv55\") pod \"node-ca-2mlqx\" (UID: \"0673f240-f3b2-45cc-bfe2-fe7d49b5cd04\") " pod="openshift-image-registry/node-ca-2mlqx" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198509 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198528 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-conf-dir\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198545 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-run-systemd\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198564 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/854a7daf-90ae-4ab2-91da-383f0f8146d3-tmp\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198580 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln26l\" (UniqueName: \"kubernetes.io/projected/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-kube-api-access-ln26l\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198594 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-system-cni-dir\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198612 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-var-lib-cni-bin\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198626 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-var-lib-kubelet\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198644 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-hostroot\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198676 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fp56\" (UniqueName: \"kubernetes.io/projected/854a7daf-90ae-4ab2-91da-383f0f8146d3-kube-api-access-6fp56\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.222365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198695 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0673f240-f3b2-45cc-bfe2-fe7d49b5cd04-serviceca\") pod \"node-ca-2mlqx\" (UID: \"0673f240-f3b2-45cc-bfe2-fe7d49b5cd04\") " pod="openshift-image-registry/node-ca-2mlqx" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198711 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198774 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-cnibin\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198832 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-run-netns\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198898 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198954 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-modprobe-d\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.198983 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199016 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199057 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-cni-bin\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199084 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6gsv\" (UniqueName: \"kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv\") pod \"network-check-target-mw4n7\" (UID: \"e79232c6-770d-4b00-80e3-43476bc0b64a\") " pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199132 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-var-lib-kubelet\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199166 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-systemd-units\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199187 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-etc-openvswitch\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.222997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199210 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/076528ce-8f31-4354-8412-26284d944989-iptables-alerter-script\") pod \"iptables-alerter-nhhtr\" (UID: \"076528ce-8f31-4354-8412-26284d944989\") " pod="openshift-network-operator/iptables-alerter-nhhtr" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199234 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-registration-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199265 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-systemd\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199309 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7207eef2-b107-462b-bcfb-6302820c4835-konnectivity-ca\") pod \"konnectivity-agent-k5k4q\" (UID: \"7207eef2-b107-462b-bcfb-6302820c4835\") " pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199352 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/08de1be9-11e6-4afd-883d-6f3e2cca1439-cni-binary-copy\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199382 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-run-multus-certs\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199425 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72394437-d6bc-4577-8e92-ce60aec0f34a-hosts-file\") pod \"node-resolver-nsvqx\" (UID: \"72394437-d6bc-4577-8e92-ce60aec0f34a\") " pod="openshift-dns/node-resolver-nsvqx" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199449 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-cni-netd\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199478 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-sysctl-conf\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199533 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-etc-kubernetes\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199556 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-run-openvswitch\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199577 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-run-ovn\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199607 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-log-socket\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199629 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae918cfe-ebcc-4398-b50c-dacff69aa164-kubelet-config\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:52.224068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199675 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-kubernetes\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199695 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-run\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199738 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-ovn-node-metrics-cert\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199836 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-run-netns\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199860 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7207eef2-b107-462b-bcfb-6302820c4835-agent-certs\") pod \"konnectivity-agent-k5k4q\" (UID: \"7207eef2-b107-462b-bcfb-6302820c4835\") " pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199907 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-system-cni-dir\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199928 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgdjr\" (UniqueName: \"kubernetes.io/projected/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-kube-api-access-fgdjr\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199960 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-etc-selinux\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.199990 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-host\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.200012 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-ovnkube-config\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.200039 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-env-overrides\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.200059 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-sysconfig\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.200079 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-os-release\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.200103 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkz4t\" (UniqueName: \"kubernetes.io/projected/08de1be9-11e6-4afd-883d-6f3e2cca1439-kube-api-access-qkz4t\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.225302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.200137 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-sysctl-d\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.226055 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.200178 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-tuned\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.226055 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.200206 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-var-lib-openvswitch\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.226055 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.205696 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-11.ec2.internal" event={"ID":"6eba61179e763a6a602c3d40734955c6","Type":"ContainerStarted","Data":"55f221590b49170244a873b2544666d1911f2f090443105ab9bc74839f4295c6"} Jul 15 18:50:52.226055 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.208985 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" event={"ID":"4e798cb305b1aaccf6fbad4a0b532fc3","Type":"ContainerStarted","Data":"7e3ce197a9585ce8022609cc8c0528755b264db9697502e2953e9c94fb091280"} Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300660 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qv55\" (UniqueName: \"kubernetes.io/projected/0673f240-f3b2-45cc-bfe2-fe7d49b5cd04-kube-api-access-7qv55\") pod \"node-ca-2mlqx\" (UID: \"0673f240-f3b2-45cc-bfe2-fe7d49b5cd04\") " pod="openshift-image-registry/node-ca-2mlqx" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300702 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300729 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-conf-dir\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300752 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-run-systemd\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300777 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/854a7daf-90ae-4ab2-91da-383f0f8146d3-tmp\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300818 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln26l\" (UniqueName: \"kubernetes.io/projected/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-kube-api-access-ln26l\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300842 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-system-cni-dir\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300868 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-var-lib-cni-bin\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300867 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-conf-dir\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300893 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-var-lib-kubelet\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300917 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-hostroot\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300952 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fp56\" (UniqueName: \"kubernetes.io/projected/854a7daf-90ae-4ab2-91da-383f0f8146d3-kube-api-access-6fp56\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300975 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0673f240-f3b2-45cc-bfe2-fe7d49b5cd04-serviceca\") pod \"node-ca-2mlqx\" (UID: \"0673f240-f3b2-45cc-bfe2-fe7d49b5cd04\") " pod="openshift-image-registry/node-ca-2mlqx" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.300999 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.331391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301021 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-cnibin\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301043 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-run-netns\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301067 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301090 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-modprobe-d\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301113 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301137 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301160 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-cni-bin\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301185 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6gsv\" (UniqueName: \"kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv\") pod \"network-check-target-mw4n7\" (UID: \"e79232c6-770d-4b00-80e3-43476bc0b64a\") " pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301631 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301631 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0673f240-f3b2-45cc-bfe2-fe7d49b5cd04-serviceca\") pod \"node-ca-2mlqx\" (UID: \"0673f240-f3b2-45cc-bfe2-fe7d49b5cd04\") " pod="openshift-image-registry/node-ca-2mlqx" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301661 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-var-lib-kubelet\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301703 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-systemd-units\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301724 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-etc-openvswitch\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301748 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/076528ce-8f31-4354-8412-26284d944989-iptables-alerter-script\") pod \"iptables-alerter-nhhtr\" (UID: \"076528ce-8f31-4354-8412-26284d944989\") " pod="openshift-network-operator/iptables-alerter-nhhtr" Jul 15 18:50:52.333715 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301773 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-registration-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301777 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-system-cni-dir\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301811 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-systemd\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301830 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-run-systemd\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301833 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7207eef2-b107-462b-bcfb-6302820c4835-konnectivity-ca\") pod \"konnectivity-agent-k5k4q\" (UID: \"7207eef2-b107-462b-bcfb-6302820c4835\") " pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301854 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/08de1be9-11e6-4afd-883d-6f3e2cca1439-cni-binary-copy\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301886 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-run-multus-certs\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301900 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301949 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-var-lib-cni-bin\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301968 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72394437-d6bc-4577-8e92-ce60aec0f34a-hosts-file\") pod \"node-resolver-nsvqx\" (UID: \"72394437-d6bc-4577-8e92-ce60aec0f34a\") " pod="openshift-dns/node-resolver-nsvqx" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301990 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-var-lib-kubelet\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302019 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-cnibin\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302026 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-hostroot\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302056 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302066 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-run-netns\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.334367 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.302174 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.302272 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret podName:ae918cfe-ebcc-4398-b50c-dacff69aa164 nodeName:}" failed. No retries permitted until 2026-07-15 18:50:52.802238583 +0000 UTC m=+3.149950878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret") pod "global-pull-secret-syncer-z65fv" (UID: "ae918cfe-ebcc-4398-b50c-dacff69aa164") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302289 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-var-lib-kubelet\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302326 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-systemd-units\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302357 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-etc-openvswitch\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.302475 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302488 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-cni-bin\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.301907 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72394437-d6bc-4577-8e92-ce60aec0f34a-hosts-file\") pod \"node-resolver-nsvqx\" (UID: \"72394437-d6bc-4577-8e92-ce60aec0f34a\") " pod="openshift-dns/node-resolver-nsvqx" Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.302531 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs podName:5ec8ecc8-2f20-485f-ab3f-738e556a7dc8 nodeName:}" failed. No retries permitted until 2026-07-15 18:50:52.802513529 +0000 UTC m=+3.150225823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs") pod "network-metrics-daemon-9669b" (UID: "5ec8ecc8-2f20-485f-ab3f-738e556a7dc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302554 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-cni-netd\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302581 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-sysctl-conf\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302605 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-etc-kubernetes\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302634 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-run-openvswitch\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302655 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-run-ovn\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.486215 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302677 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-log-socket\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302700 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae918cfe-ebcc-4398-b50c-dacff69aa164-kubelet-config\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302725 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-kubernetes\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302745 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-run\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302766 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-ovn-node-metrics-cert\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302806 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-run-netns\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302830 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7207eef2-b107-462b-bcfb-6302820c4835-agent-certs\") pod \"konnectivity-agent-k5k4q\" (UID: \"7207eef2-b107-462b-bcfb-6302820c4835\") " pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302852 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-system-cni-dir\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302876 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgdjr\" (UniqueName: \"kubernetes.io/projected/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-kube-api-access-fgdjr\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302900 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-etc-selinux\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302901 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/076528ce-8f31-4354-8412-26284d944989-iptables-alerter-script\") pod \"iptables-alerter-nhhtr\" (UID: \"076528ce-8f31-4354-8412-26284d944989\") " pod="openshift-network-operator/iptables-alerter-nhhtr" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302922 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-host\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302965 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7207eef2-b107-462b-bcfb-6302820c4835-konnectivity-ca\") pod \"konnectivity-agent-k5k4q\" (UID: \"7207eef2-b107-462b-bcfb-6302820c4835\") " pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.302965 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-run-multus-certs\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.489570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303027 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-ovnkube-config\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303032 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-modprobe-d\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303055 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-systemd\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303055 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-env-overrides\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303092 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-sysconfig\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303098 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/08de1be9-11e6-4afd-883d-6f3e2cca1439-cni-binary-copy\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303105 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-registration-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303116 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-os-release\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303135 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-cni-netd\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303140 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-host\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303140 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkz4t\" (UniqueName: \"kubernetes.io/projected/08de1be9-11e6-4afd-883d-6f3e2cca1439-kube-api-access-qkz4t\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303170 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-sysctl-d\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303194 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-tuned\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303228 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-var-lib-openvswitch\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303234 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-run\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.490340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303252 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-node-log\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303275 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae918cfe-ebcc-4398-b50c-dacff69aa164-dbus\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303283 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-sysctl-conf\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303274 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-system-cni-dir\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303336 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-etc-kubernetes\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303378 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-run-openvswitch\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303417 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-run-ovn\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303469 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-lib-modules\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303486 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae918cfe-ebcc-4398-b50c-dacff69aa164-kubelet-config\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303498 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt4wc\" (UniqueName: \"kubernetes.io/projected/72394437-d6bc-4577-8e92-ce60aec0f34a-kube-api-access-xt4wc\") pod \"node-resolver-nsvqx\" (UID: \"72394437-d6bc-4577-8e92-ce60aec0f34a\") " pod="openshift-dns/node-resolver-nsvqx" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303524 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-kubelet\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303567 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-etc-selinux\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303570 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303609 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.491429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303603 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-device-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303645 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-cnibin\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303656 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-device-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303669 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-run-k8s-cni-cncf-io\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303688 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-log-socket\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303693 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72394437-d6bc-4577-8e92-ce60aec0f34a-tmp-dir\") pod \"node-resolver-nsvqx\" (UID: \"72394437-d6bc-4577-8e92-ce60aec0f34a\") " pod="openshift-dns/node-resolver-nsvqx" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303729 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/076528ce-8f31-4354-8412-26284d944989-host-slash\") pod \"iptables-alerter-nhhtr\" (UID: \"076528ce-8f31-4354-8412-26284d944989\") " pod="openshift-network-operator/iptables-alerter-nhhtr" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303761 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-kubelet\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303771 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-ovnkube-config\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303772 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppfpx\" (UniqueName: \"kubernetes.io/projected/f95dfaa6-6b45-4541-8827-af5e7ab0f492-kube-api-access-ppfpx\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303835 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-var-lib-cni-multus\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303860 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-lib-modules\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303868 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-run-k8s-cni-cncf-io\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303860 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-slash\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303911 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-ovnkube-script-lib\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.492459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303919 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-slash\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303938 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-sys-fs\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303964 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-sys\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303992 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-os-release\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303999 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/076528ce-8f31-4354-8412-26284d944989-host-slash\") pod \"iptables-alerter-nhhtr\" (UID: \"076528ce-8f31-4354-8412-26284d944989\") " pod="openshift-network-operator/iptables-alerter-nhhtr" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304017 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-socket-dir-parent\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304043 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-run-ovn-kubernetes\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304071 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlczt\" (UniqueName: \"kubernetes.io/projected/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-kube-api-access-nlczt\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304095 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w45c6\" (UniqueName: \"kubernetes.io/projected/076528ce-8f31-4354-8412-26284d944989-kube-api-access-w45c6\") pod \"iptables-alerter-nhhtr\" (UID: \"076528ce-8f31-4354-8412-26284d944989\") " pod="openshift-network-operator/iptables-alerter-nhhtr" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304119 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-socket-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304126 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72394437-d6bc-4577-8e92-ce60aec0f34a-tmp-dir\") pod \"node-resolver-nsvqx\" (UID: \"72394437-d6bc-4577-8e92-ce60aec0f34a\") " pod="openshift-dns/node-resolver-nsvqx" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304146 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304173 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-cni-dir\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304253 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-daemon-config\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.493728 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304314 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-socket-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304341 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0673f240-f3b2-45cc-bfe2-fe7d49b5cd04-host\") pod \"node-ca-2mlqx\" (UID: \"0673f240-f3b2-45cc-bfe2-fe7d49b5cd04\") " pod="openshift-image-registry/node-ca-2mlqx" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.303169 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-kubernetes\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304422 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0673f240-f3b2-45cc-bfe2-fe7d49b5cd04-host\") pod \"node-ca-2mlqx\" (UID: \"0673f240-f3b2-45cc-bfe2-fe7d49b5cd04\") " pod="openshift-image-registry/node-ca-2mlqx" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304422 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-sys-fs\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304452 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304460 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-sys\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304493 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-run-netns\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304531 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-host-run-ovn-kubernetes\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304565 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-cni-dir\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304620 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-os-release\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304660 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-socket-dir-parent\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304675 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-node-log\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304691 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-cnibin\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.494586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304771 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-var-lib-openvswitch\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.304859 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.305000 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-sysconfig\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.305055 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae918cfe-ebcc-4398-b50c-dacff69aa164-dbus\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.305081 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-env-overrides\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.305089 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-host-var-lib-cni-multus\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.305106 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-ovnkube-script-lib\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.305140 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/08de1be9-11e6-4afd-883d-6f3e2cca1439-os-release\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.305183 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f95dfaa6-6b45-4541-8827-af5e7ab0f492-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.305186 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-sysctl-d\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.305746 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/08de1be9-11e6-4afd-883d-6f3e2cca1439-multus-daemon-config\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.307850 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7207eef2-b107-462b-bcfb-6302820c4835-agent-certs\") pod \"konnectivity-agent-k5k4q\" (UID: \"7207eef2-b107-462b-bcfb-6302820c4835\") " pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.314397 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/854a7daf-90ae-4ab2-91da-383f0f8146d3-tmp\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.314435 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/854a7daf-90ae-4ab2-91da-383f0f8146d3-etc-tuned\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.314449 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-ovn-node-metrics-cert\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.314966 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:50:52.495637 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.314985 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.315000 2574 projected.go:196] Error preparing data for projected volume kube-api-access-g6gsv for pod openshift-network-diagnostics/network-check-target-mw4n7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.315057 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv podName:e79232c6-770d-4b00-80e3-43476bc0b64a nodeName:}" failed. No retries permitted until 2026-07-15 18:50:52.815040382 +0000 UTC m=+3.162752682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g6gsv" (UniqueName: "kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv") pod "network-check-target-mw4n7" (UID: "e79232c6-770d-4b00-80e3-43476bc0b64a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.319185 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fp56\" (UniqueName: \"kubernetes.io/projected/854a7daf-90ae-4ab2-91da-383f0f8146d3-kube-api-access-6fp56\") pod \"tuned-m8xkn\" (UID: \"854a7daf-90ae-4ab2-91da-383f0f8146d3\") " pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.319215 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qv55\" (UniqueName: \"kubernetes.io/projected/0673f240-f3b2-45cc-bfe2-fe7d49b5cd04-kube-api-access-7qv55\") pod \"node-ca-2mlqx\" (UID: \"0673f240-f3b2-45cc-bfe2-fe7d49b5cd04\") " pod="openshift-image-registry/node-ca-2mlqx" Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.319591 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgdjr\" (UniqueName: \"kubernetes.io/projected/4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6-kube-api-access-fgdjr\") pod \"multus-additional-cni-plugins-xcdnw\" (UID: \"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6\") " pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.328178 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln26l\" (UniqueName: \"kubernetes.io/projected/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-kube-api-access-ln26l\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.338930 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlczt\" (UniqueName: \"kubernetes.io/projected/e241c2dc-dedc-4a81-8e1b-6a5488efeddb-kube-api-access-nlczt\") pod \"ovnkube-node-cjxwn\" (UID: \"e241c2dc-dedc-4a81-8e1b-6a5488efeddb\") " pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.339383 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt4wc\" (UniqueName: \"kubernetes.io/projected/72394437-d6bc-4577-8e92-ce60aec0f34a-kube-api-access-xt4wc\") pod \"node-resolver-nsvqx\" (UID: \"72394437-d6bc-4577-8e92-ce60aec0f34a\") " pod="openshift-dns/node-resolver-nsvqx" Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.340249 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppfpx\" (UniqueName: \"kubernetes.io/projected/f95dfaa6-6b45-4541-8827-af5e7ab0f492-kube-api-access-ppfpx\") pod \"aws-ebs-csi-driver-node-l9ggd\" (UID: \"f95dfaa6-6b45-4541-8827-af5e7ab0f492\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.340940 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkz4t\" (UniqueName: \"kubernetes.io/projected/08de1be9-11e6-4afd-883d-6f3e2cca1439-kube-api-access-qkz4t\") pod \"multus-qc4kh\" (UID: \"08de1be9-11e6-4afd-883d-6f3e2cca1439\") " pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.341497 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w45c6\" (UniqueName: \"kubernetes.io/projected/076528ce-8f31-4354-8412-26284d944989-kube-api-access-w45c6\") pod \"iptables-alerter-nhhtr\" (UID: \"076528ce-8f31-4354-8412-26284d944989\") " pod="openshift-network-operator/iptables-alerter-nhhtr" Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.405248 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nsvqx" Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.420220 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2mlqx" Jul 15 18:50:52.497374 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:52.420760 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72394437_d6bc_4577_8e92_ce60aec0f34a.slice/crio-ef9aa235d2670d5124397adb6a074061a8287784ade90adf47254a3a6ad3d791 WatchSource:0}: Error finding container ef9aa235d2670d5124397adb6a074061a8287784ade90adf47254a3a6ad3d791: Status 404 returned error can't find the container with id ef9aa235d2670d5124397adb6a074061a8287784ade90adf47254a3a6ad3d791 Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:52.429916 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0673f240_f3b2_45cc_bfe2_fe7d49b5cd04.slice/crio-48c02ca0836e90022838a21f458af19eb7f75a78693bf32c0f8ee97adf44284d WatchSource:0}: Error finding container 48c02ca0836e90022838a21f458af19eb7f75a78693bf32c0f8ee97adf44284d: Status 404 returned error can't find the container with id 48c02ca0836e90022838a21f458af19eb7f75a78693bf32c0f8ee97adf44284d Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.431300 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.438924 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:52.442081 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode241c2dc_dedc_4a81_8e1b_6a5488efeddb.slice/crio-690fc0b90bd461317e10d6b347d7d9b415e689363d1a3d6b3f9113df5ce9ece1 WatchSource:0}: Error finding container 690fc0b90bd461317e10d6b347d7d9b415e689363d1a3d6b3f9113df5ce9ece1: Status 404 returned error can't find the container with id 690fc0b90bd461317e10d6b347d7d9b415e689363d1a3d6b3f9113df5ce9ece1 Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.445329 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xcdnw" Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:52.452565 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7207eef2_b107_462b_bcfb_6302820c4835.slice/crio-085fe08ef5207797c59789851c31aed6f175cba0f894c251f6081129fe41b949 WatchSource:0}: Error finding container 085fe08ef5207797c59789851c31aed6f175cba0f894c251f6081129fe41b949: Status 404 returned error can't find the container with id 085fe08ef5207797c59789851c31aed6f175cba0f894c251f6081129fe41b949 Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.454098 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.454185 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qc4kh" Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:52.458468 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c89d0f4_91c2_493b_a3a8_e0ee2c6e41e6.slice/crio-b58f3df0b9502ecb9be0f4c2aa1c434884ff88ced4c8a211c7fe177acde125db WatchSource:0}: Error finding container b58f3df0b9502ecb9be0f4c2aa1c434884ff88ced4c8a211c7fe177acde125db: Status 404 returned error can't find the container with id b58f3df0b9502ecb9be0f4c2aa1c434884ff88ced4c8a211c7fe177acde125db Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:52.463518 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08de1be9_11e6_4afd_883d_6f3e2cca1439.slice/crio-36e8c9aa532e0a98621a63162defc41b1e0bfbd3affc924cfad91e649d909fb0 WatchSource:0}: Error finding container 36e8c9aa532e0a98621a63162defc41b1e0bfbd3affc924cfad91e649d909fb0: Status 404 returned error can't find the container with id 36e8c9aa532e0a98621a63162defc41b1e0bfbd3affc924cfad91e649d909fb0 Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.479128 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nhhtr" Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.487874 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" Jul 15 18:50:52.498013 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.492911 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" Jul 15 18:50:52.517599 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:52.517571 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076528ce_8f31_4354_8412_26284d944989.slice/crio-876827714a030bcd15e186237bb4bda2f1c198e7bff45fda048f24e2391de90f WatchSource:0}: Error finding container 876827714a030bcd15e186237bb4bda2f1c198e7bff45fda048f24e2391de90f: Status 404 returned error can't find the container with id 876827714a030bcd15e186237bb4bda2f1c198e7bff45fda048f24e2391de90f Jul 15 18:50:52.527589 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:52.527562 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95dfaa6_6b45_4541_8827_af5e7ab0f492.slice/crio-3f5c461087ef056c447c62767b7402b3bc1eab80f68dd41ab42eb45dc2d81f1f WatchSource:0}: Error finding container 3f5c461087ef056c447c62767b7402b3bc1eab80f68dd41ab42eb45dc2d81f1f: Status 404 returned error can't find the container with id 3f5c461087ef056c447c62767b7402b3bc1eab80f68dd41ab42eb45dc2d81f1f Jul 15 18:50:52.533972 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:50:52.533944 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod854a7daf_90ae_4ab2_91da_383f0f8146d3.slice/crio-e32b92e088a4b81d55f2d76bd96298bea63af19c25e755e53624e1c3c13a121a WatchSource:0}: Error finding container e32b92e088a4b81d55f2d76bd96298bea63af19c25e755e53624e1c3c13a121a: Status 404 returned error can't find the container with id e32b92e088a4b81d55f2d76bd96298bea63af19c25e755e53624e1c3c13a121a Jul 15 18:50:52.810037 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.809873 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:52.810037 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.809986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:52.810037 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.810000 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:50:52.810279 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.810078 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs podName:5ec8ecc8-2f20-485f-ab3f-738e556a7dc8 nodeName:}" failed. No retries permitted until 2026-07-15 18:50:53.810058296 +0000 UTC m=+4.157770586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs") pod "network-metrics-daemon-9669b" (UID: "5ec8ecc8-2f20-485f-ab3f-738e556a7dc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:50:52.810279 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.810092 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:50:52.810279 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.810142 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret podName:ae918cfe-ebcc-4398-b50c-dacff69aa164 nodeName:}" failed. No retries permitted until 2026-07-15 18:50:53.810126326 +0000 UTC m=+4.157838626 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret") pod "global-pull-secret-syncer-z65fv" (UID: "ae918cfe-ebcc-4398-b50c-dacff69aa164") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:50:52.911231 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:52.911191 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6gsv\" (UniqueName: \"kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv\") pod \"network-check-target-mw4n7\" (UID: \"e79232c6-770d-4b00-80e3-43476bc0b64a\") " pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:50:52.911391 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.911369 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:50:52.911435 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.911395 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:50:52.911435 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.911405 2574 projected.go:196] Error preparing data for projected volume kube-api-access-g6gsv for pod openshift-network-diagnostics/network-check-target-mw4n7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:50:52.911508 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:52.911456 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv podName:e79232c6-770d-4b00-80e3-43476bc0b64a nodeName:}" failed. No retries permitted until 2026-07-15 18:50:53.911442256 +0000 UTC m=+4.259154532 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g6gsv" (UniqueName: "kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv") pod "network-check-target-mw4n7" (UID: "e79232c6-770d-4b00-80e3-43476bc0b64a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:50:53.023511 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.023475 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:50:53.154151 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.154035 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 18:45:52 +0000 UTC" deadline="2028-04-03 20:49:29.075077314 +0000 UTC" Jul 15 18:50:53.154151 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.154104 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15073h58m35.920977093s" Jul 15 18:50:53.201286 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.200799 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:53.201286 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:53.200936 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:50:53.212936 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.212890 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" event={"ID":"f95dfaa6-6b45-4541-8827-af5e7ab0f492","Type":"ContainerStarted","Data":"3f5c461087ef056c447c62767b7402b3bc1eab80f68dd41ab42eb45dc2d81f1f"} Jul 15 18:50:53.214362 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.214310 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nhhtr" event={"ID":"076528ce-8f31-4354-8412-26284d944989","Type":"ContainerStarted","Data":"876827714a030bcd15e186237bb4bda2f1c198e7bff45fda048f24e2391de90f"} Jul 15 18:50:53.219240 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.219172 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2mlqx" event={"ID":"0673f240-f3b2-45cc-bfe2-fe7d49b5cd04","Type":"ContainerStarted","Data":"48c02ca0836e90022838a21f458af19eb7f75a78693bf32c0f8ee97adf44284d"} Jul 15 18:50:53.221153 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.221129 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" event={"ID":"854a7daf-90ae-4ab2-91da-383f0f8146d3","Type":"ContainerStarted","Data":"e32b92e088a4b81d55f2d76bd96298bea63af19c25e755e53624e1c3c13a121a"} Jul 15 18:50:53.223665 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.223643 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qc4kh" event={"ID":"08de1be9-11e6-4afd-883d-6f3e2cca1439","Type":"ContainerStarted","Data":"36e8c9aa532e0a98621a63162defc41b1e0bfbd3affc924cfad91e649d909fb0"} Jul 15 18:50:53.229155 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.229130 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcdnw" event={"ID":"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6","Type":"ContainerStarted","Data":"b58f3df0b9502ecb9be0f4c2aa1c434884ff88ced4c8a211c7fe177acde125db"} Jul 15 18:50:53.231139 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.231112 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k5k4q" event={"ID":"7207eef2-b107-462b-bcfb-6302820c4835","Type":"ContainerStarted","Data":"085fe08ef5207797c59789851c31aed6f175cba0f894c251f6081129fe41b949"} Jul 15 18:50:53.234158 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.234126 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" event={"ID":"e241c2dc-dedc-4a81-8e1b-6a5488efeddb","Type":"ContainerStarted","Data":"690fc0b90bd461317e10d6b347d7d9b415e689363d1a3d6b3f9113df5ce9ece1"} Jul 15 18:50:53.235546 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.235498 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nsvqx" event={"ID":"72394437-d6bc-4577-8e92-ce60aec0f34a","Type":"ContainerStarted","Data":"ef9aa235d2670d5124397adb6a074061a8287784ade90adf47254a3a6ad3d791"} Jul 15 18:50:53.819254 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.819217 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:53.819438 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.819270 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:53.819438 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:53.819397 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:50:53.819547 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:53.819436 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:50:53.819547 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:53.819458 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs podName:5ec8ecc8-2f20-485f-ab3f-738e556a7dc8 nodeName:}" failed. No retries permitted until 2026-07-15 18:50:55.819438711 +0000 UTC m=+6.167150988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs") pod "network-metrics-daemon-9669b" (UID: "5ec8ecc8-2f20-485f-ab3f-738e556a7dc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:50:53.819547 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:53.819530 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret podName:ae918cfe-ebcc-4398-b50c-dacff69aa164 nodeName:}" failed. No retries permitted until 2026-07-15 18:50:55.819510723 +0000 UTC m=+6.167223009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret") pod "global-pull-secret-syncer-z65fv" (UID: "ae918cfe-ebcc-4398-b50c-dacff69aa164") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:50:53.919639 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:53.919603 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6gsv\" (UniqueName: \"kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv\") pod \"network-check-target-mw4n7\" (UID: \"e79232c6-770d-4b00-80e3-43476bc0b64a\") " pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:50:53.919857 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:53.919816 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:50:53.919857 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:53.919836 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:50:53.919857 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:53.919848 2574 projected.go:196] Error preparing data for projected volume kube-api-access-g6gsv for pod openshift-network-diagnostics/network-check-target-mw4n7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:50:53.920018 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:53.919904 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv podName:e79232c6-770d-4b00-80e3-43476bc0b64a nodeName:}" failed. No retries permitted until 2026-07-15 18:50:55.919886296 +0000 UTC m=+6.267598590 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g6gsv" (UniqueName: "kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv") pod "network-check-target-mw4n7" (UID: "e79232c6-770d-4b00-80e3-43476bc0b64a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:50:54.154843 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:54.154743 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 18:45:52 +0000 UTC" deadline="2028-04-26 23:46:34.943813206 +0000 UTC" Jul 15 18:50:54.154843 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:54.154801 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15628h55m40.789017455s" Jul 15 18:50:54.201306 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:54.201277 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:54.201645 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:54.201415 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:50:54.201868 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:54.201847 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:50:54.201968 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:54.201945 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:50:55.200733 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:55.200278 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:55.200733 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:55.200400 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:50:55.838196 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:55.837598 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:55.838196 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:55.837649 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:55.838196 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:55.837809 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:50:55.838196 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:55.837869 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs podName:5ec8ecc8-2f20-485f-ab3f-738e556a7dc8 nodeName:}" failed. No retries permitted until 2026-07-15 18:50:59.837851111 +0000 UTC m=+10.185563401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs") pod "network-metrics-daemon-9669b" (UID: "5ec8ecc8-2f20-485f-ab3f-738e556a7dc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:50:55.838196 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:55.837887 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:50:55.838196 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:55.837952 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret podName:ae918cfe-ebcc-4398-b50c-dacff69aa164 nodeName:}" failed. No retries permitted until 2026-07-15 18:50:59.837934392 +0000 UTC m=+10.185646691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret") pod "global-pull-secret-syncer-z65fv" (UID: "ae918cfe-ebcc-4398-b50c-dacff69aa164") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:50:55.939014 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:55.938978 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6gsv\" (UniqueName: \"kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv\") pod \"network-check-target-mw4n7\" (UID: \"e79232c6-770d-4b00-80e3-43476bc0b64a\") " pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:50:55.939192 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:55.939175 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:50:55.939263 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:55.939200 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:50:55.939263 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:55.939213 2574 projected.go:196] Error preparing data for projected volume kube-api-access-g6gsv for pod openshift-network-diagnostics/network-check-target-mw4n7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:50:55.939364 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:55.939269 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv podName:e79232c6-770d-4b00-80e3-43476bc0b64a nodeName:}" failed. No retries permitted until 2026-07-15 18:50:59.939250843 +0000 UTC m=+10.286963142 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g6gsv" (UniqueName: "kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv") pod "network-check-target-mw4n7" (UID: "e79232c6-770d-4b00-80e3-43476bc0b64a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:50:56.201439 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:56.201322 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:50:56.201627 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:56.201460 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:50:56.201627 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:56.201538 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:56.201733 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:56.201653 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:50:57.201014 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:57.200976 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:57.201435 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:57.201115 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:50:58.200503 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:58.200314 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:50:58.200503 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:58.200491 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:58.200721 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:58.200590 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:50:58.201060 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:58.201017 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:50:59.201421 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:59.200934 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:59.201421 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:59.201071 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:50:59.872090 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:59.872052 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:50:59.872241 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:59.872105 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:50:59.872279 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:59.872244 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:50:59.872662 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:59.872310 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs podName:5ec8ecc8-2f20-485f-ab3f-738e556a7dc8 nodeName:}" failed. No retries permitted until 2026-07-15 18:51:07.872287734 +0000 UTC m=+18.220000013 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs") pod "network-metrics-daemon-9669b" (UID: "5ec8ecc8-2f20-485f-ab3f-738e556a7dc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:50:59.872662 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:59.872552 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:50:59.872662 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:59.872606 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret podName:ae918cfe-ebcc-4398-b50c-dacff69aa164 nodeName:}" failed. No retries permitted until 2026-07-15 18:51:07.872588989 +0000 UTC m=+18.220301281 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret") pod "global-pull-secret-syncer-z65fv" (UID: "ae918cfe-ebcc-4398-b50c-dacff69aa164") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:50:59.972953 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:50:59.972901 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6gsv\" (UniqueName: \"kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv\") pod \"network-check-target-mw4n7\" (UID: \"e79232c6-770d-4b00-80e3-43476bc0b64a\") " pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:50:59.973109 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:59.973074 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:50:59.973109 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:59.973095 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:50:59.973109 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:59.973107 2574 projected.go:196] Error preparing data for projected volume kube-api-access-g6gsv for pod openshift-network-diagnostics/network-check-target-mw4n7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:50:59.973266 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:50:59.973162 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv podName:e79232c6-770d-4b00-80e3-43476bc0b64a nodeName:}" failed. No retries permitted until 2026-07-15 18:51:07.973143935 +0000 UTC m=+18.320856225 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g6gsv" (UniqueName: "kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv") pod "network-check-target-mw4n7" (UID: "e79232c6-770d-4b00-80e3-43476bc0b64a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:51:00.202174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:00.201509 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:00.202174 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:00.201626 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:00.202174 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:00.201520 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:00.202174 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:00.202059 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:01.201108 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:01.201062 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:01.201300 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:01.201201 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:02.200831 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:02.200778 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:02.201274 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:02.200796 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:02.201274 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:02.200938 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:02.201274 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:02.201022 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:03.200602 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:03.200515 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:03.200750 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:03.200664 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:04.200412 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:04.200383 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:04.200895 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:04.200383 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:04.200895 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:04.200509 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:04.200895 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:04.200632 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:05.200142 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:05.200109 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:05.200339 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:05.200233 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:06.200370 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:06.200335 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:06.200772 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:06.200339 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:06.200772 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:06.200453 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:06.200772 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:06.200560 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:07.200250 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:07.200216 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:07.200460 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:07.200329 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:07.933367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:07.933332 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:07.933367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:07.933369 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:07.933586 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:07.933550 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:51:07.933638 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:07.933625 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs podName:5ec8ecc8-2f20-485f-ab3f-738e556a7dc8 nodeName:}" failed. No retries permitted until 2026-07-15 18:51:23.933603059 +0000 UTC m=+34.281315356 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs") pod "network-metrics-daemon-9669b" (UID: "5ec8ecc8-2f20-485f-ab3f-738e556a7dc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:51:07.933690 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:07.933550 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:51:07.933726 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:07.933712 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret podName:ae918cfe-ebcc-4398-b50c-dacff69aa164 nodeName:}" failed. No retries permitted until 2026-07-15 18:51:23.933683149 +0000 UTC m=+34.281395446 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret") pod "global-pull-secret-syncer-z65fv" (UID: "ae918cfe-ebcc-4398-b50c-dacff69aa164") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:51:08.033850 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:08.033818 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6gsv\" (UniqueName: \"kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv\") pod \"network-check-target-mw4n7\" (UID: \"e79232c6-770d-4b00-80e3-43476bc0b64a\") " pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:08.034034 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:08.033981 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:51:08.034034 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:08.034000 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:51:08.034034 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:08.034011 2574 projected.go:196] Error preparing data for projected volume kube-api-access-g6gsv for pod openshift-network-diagnostics/network-check-target-mw4n7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:51:08.034188 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:08.034064 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv podName:e79232c6-770d-4b00-80e3-43476bc0b64a nodeName:}" failed. No retries permitted until 2026-07-15 18:51:24.03404796 +0000 UTC m=+34.381760256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g6gsv" (UniqueName: "kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv") pod "network-check-target-mw4n7" (UID: "e79232c6-770d-4b00-80e3-43476bc0b64a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:51:08.200938 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:08.200850 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:08.200938 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:08.200893 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:08.201433 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:08.200998 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:08.201495 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:08.201419 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:09.201221 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:09.201184 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:09.201645 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:09.201330 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:10.201605 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:10.201563 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:10.202068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:10.201659 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:10.202068 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:10.201697 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:10.202068 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:10.201803 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:11.200189 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:11.200052 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:11.200285 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:11.200267 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:11.279066 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:11.279034 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-11.ec2.internal" event={"ID":"6eba61179e763a6a602c3d40734955c6","Type":"ContainerStarted","Data":"04cd3a335c1eb3fd0a22706ad1caf996c7f0a85b6aa1f39e43ea1eb94bead7cf"} Jul 15 18:51:11.280428 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:11.280410 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" event={"ID":"854a7daf-90ae-4ab2-91da-383f0f8146d3","Type":"ContainerStarted","Data":"27b31ea2cf02085ea7f17a7598c695351a0dd8a155d3cb9155676319add944d2"} Jul 15 18:51:11.292904 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:11.292857 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-11.ec2.internal" podStartSLOduration=20.292844813 podStartE2EDuration="20.292844813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:11.292434504 +0000 UTC m=+21.640146803" watchObservedRunningTime="2026-07-15 18:51:11.292844813 +0000 UTC m=+21.640557113" Jul 15 18:51:11.307546 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:11.307510 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m8xkn" podStartSLOduration=2.787070532 podStartE2EDuration="21.307497275s" totalImagesPullingTime="18.520426743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:11.306898603 +0000 UTC m=+21.654610895" watchObservedRunningTime="2026-07-15 18:51:11.307497275 +0000 UTC m=+21.655209572" Jul 15 18:51:12.200924 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.200660 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:12.201075 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.200733 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:12.201075 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:12.200988 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:12.201075 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:12.201061 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:12.284357 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.284328 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qc4kh" event={"ID":"08de1be9-11e6-4afd-883d-6f3e2cca1439","Type":"ContainerStarted","Data":"b42cdf21846e225f75b558253d9eec34716418c3d319965951ac8f11ca448b9a"} Jul 15 18:51:12.286133 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.286110 2574 generic.go:356] "Generic (PLEG): container finished" podID="4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6" containerID="d60a380156c6901bc04f45bedef9a8d94144983fab56450348c6e6d37235c480" exitCode=0 Jul 15 18:51:12.286253 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.286143 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcdnw" event={"ID":"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6","Type":"ContainerDied","Data":"d60a380156c6901bc04f45bedef9a8d94144983fab56450348c6e6d37235c480"} Jul 15 18:51:12.288159 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.288137 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k5k4q" event={"ID":"7207eef2-b107-462b-bcfb-6302820c4835","Type":"ContainerStarted","Data":"dedb82c8b109cd1d31ac6c8799c136f5263cebca234ed130c275fbb9bf949a37"} Jul 15 18:51:12.291129 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.291112 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 18:51:12.291491 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.291467 2574 generic.go:356] "Generic (PLEG): container finished" podID="e241c2dc-dedc-4a81-8e1b-6a5488efeddb" containerID="904ddd3214b2302d3a562dd35476f8748fd67ca76425cc18579de0153d2774a3" exitCode=1 Jul 15 18:51:12.291577 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.291534 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" event={"ID":"e241c2dc-dedc-4a81-8e1b-6a5488efeddb","Type":"ContainerStarted","Data":"d16f65fd64aeb832b95b247df2cc00d5e98b53a0bd79c92fd5102ac04034c3ee"} Jul 15 18:51:12.291632 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.291609 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" event={"ID":"e241c2dc-dedc-4a81-8e1b-6a5488efeddb","Type":"ContainerStarted","Data":"d7bfd51565b5f5c50e9d67fb26a908a7a9aaa97ba50ea55990c353cb9d5a5304"} Jul 15 18:51:12.291686 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.291660 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" event={"ID":"e241c2dc-dedc-4a81-8e1b-6a5488efeddb","Type":"ContainerStarted","Data":"24f4ea7912583267458b148f8c416be321a3f02428de0cf3226eba45b17ed002"} Jul 15 18:51:12.291742 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.291707 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" event={"ID":"e241c2dc-dedc-4a81-8e1b-6a5488efeddb","Type":"ContainerStarted","Data":"3731368e987e540425f076191df579a1c323040e91c8cd6219f68ee0d943a1b9"} Jul 15 18:51:12.291832 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.291803 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" event={"ID":"e241c2dc-dedc-4a81-8e1b-6a5488efeddb","Type":"ContainerDied","Data":"904ddd3214b2302d3a562dd35476f8748fd67ca76425cc18579de0153d2774a3"} Jul 15 18:51:12.291884 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.291859 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" event={"ID":"e241c2dc-dedc-4a81-8e1b-6a5488efeddb","Type":"ContainerStarted","Data":"88f6a2e4e5139fb73dc2dea58cf32985171365b9e42324dc0adacc4c834971e3"} Jul 15 18:51:12.293130 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.293099 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nsvqx" event={"ID":"72394437-d6bc-4577-8e92-ce60aec0f34a","Type":"ContainerStarted","Data":"3484a644c052b0650613a3c4fb5a6e2adb86c8e5e6c2769cc6658685163f9f71"} Jul 15 18:51:12.294712 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.294691 2574 generic.go:356] "Generic (PLEG): container finished" podID="4e798cb305b1aaccf6fbad4a0b532fc3" containerID="170fbca35fc01420d79883653340297fca240d2f664b74e9707bea4239e53b73" exitCode=0 Jul 15 18:51:12.294820 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.294775 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" event={"ID":"4e798cb305b1aaccf6fbad4a0b532fc3","Type":"ContainerDied","Data":"170fbca35fc01420d79883653340297fca240d2f664b74e9707bea4239e53b73"} Jul 15 18:51:12.296350 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.296328 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" event={"ID":"f95dfaa6-6b45-4541-8827-af5e7ab0f492","Type":"ContainerStarted","Data":"9474dd18c50db75a78fb6d8d030f6c498677c16cc91e23432bb4379d3bfe7177"} Jul 15 18:51:12.297863 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.297838 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2mlqx" event={"ID":"0673f240-f3b2-45cc-bfe2-fe7d49b5cd04","Type":"ContainerStarted","Data":"b83da253d74939ffb05a801bd734c58babaa787df6cab81a75967011d8e248f3"} Jul 15 18:51:12.306162 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.306127 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qc4kh" podStartSLOduration=3.427875196 podStartE2EDuration="22.306111687s" totalImagesPullingTime="18.878236491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:12.305535572 +0000 UTC m=+22.653247870" watchObservedRunningTime="2026-07-15 18:51:12.306111687 +0000 UTC m=+22.653823981" Jul 15 18:51:12.320649 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.320612 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2mlqx" podStartSLOduration=8.249618442 podStartE2EDuration="22.320600746s" totalImagesPullingTime="14.070982304s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:12.32027558 +0000 UTC m=+22.667987879" watchObservedRunningTime="2026-07-15 18:51:12.320600746 +0000 UTC m=+22.668313044" Jul 15 18:51:12.344511 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.344469 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-k5k4q" podStartSLOduration=3.961278975 podStartE2EDuration="22.344451357s" totalImagesPullingTime="18.383172382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:12.344164311 +0000 UTC m=+22.691876611" watchObservedRunningTime="2026-07-15 18:51:12.344451357 +0000 UTC m=+22.692163656" Jul 15 18:51:12.418531 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.418482 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nsvqx" podStartSLOduration=3.784117908 podStartE2EDuration="22.418462711s" totalImagesPullingTime="18.634344803s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:12.418214608 +0000 UTC m=+22.765926906" watchObservedRunningTime="2026-07-15 18:51:12.418462711 +0000 UTC m=+22.766175010" Jul 15 18:51:12.434962 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.434924 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:12.520779 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.520753 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:12.583474 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:12.583443 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:13.026358 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.026335 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 18:51:13.146013 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.145910 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T18:51:13.026354576Z","UUID":"3fdde70a-e1f6-4ae0-a1e0-561c42c08cfe","Handler":null,"Name":"","Endpoint":""} Jul 15 18:51:13.147938 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.147886 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 18:51:13.147938 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.147921 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 18:51:13.201057 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.201031 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:13.201202 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:13.201133 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:13.302316 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.302283 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" event={"ID":"f95dfaa6-6b45-4541-8827-af5e7ab0f492","Type":"ContainerStarted","Data":"d2efa9ef5a61c66173e06b10304331b56cb17ebf80390a29e9da75c7eb97e7cd"} Jul 15 18:51:13.304064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.304028 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nhhtr" event={"ID":"076528ce-8f31-4354-8412-26284d944989","Type":"ContainerStarted","Data":"7679d28f56d988be5755f1ef4e61d38cb8cc09c99e0957de651802205f1cea19"} Jul 15 18:51:13.305987 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.305959 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" event={"ID":"4e798cb305b1aaccf6fbad4a0b532fc3","Type":"ContainerStarted","Data":"9dbe6945d37097f32fac29a7683fb1c385b6a1497b53a3ecd3b296513f87109a"} Jul 15 18:51:13.320186 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.320143 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nhhtr" podStartSLOduration=4.785608942 podStartE2EDuration="23.320131419s" totalImagesPullingTime="18.534522477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:13.31981773 +0000 UTC m=+23.667530026" watchObservedRunningTime="2026-07-15 18:51:13.320131419 +0000 UTC m=+23.667843717" Jul 15 18:51:13.333048 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.333012 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-11.ec2.internal" podStartSLOduration=22.333003801 podStartE2EDuration="22.333003801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:13.3328421 +0000 UTC m=+23.680554399" watchObservedRunningTime="2026-07-15 18:51:13.333003801 +0000 UTC m=+23.680716098" Jul 15 18:51:13.419149 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.419078 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:13.488184 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.488152 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:13.885598 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.885469 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:13.987532 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:13.987497 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:14.200933 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:14.200908 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:14.201094 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:14.200904 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:14.201094 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:14.201047 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:14.201195 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:14.201117 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:14.311364 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:14.311323 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 18:51:14.311858 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:14.311777 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" event={"ID":"e241c2dc-dedc-4a81-8e1b-6a5488efeddb","Type":"ContainerStarted","Data":"32c3de4b306a9faea466a59dc5ecb3528b42cb4fe11938c30c2474b24a4112f4"} Jul 15 18:51:15.200461 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:15.200430 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:15.200644 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:15.200527 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:15.268699 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:15.268675 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:51:15.269272 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:15.269251 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:51:15.316633 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:15.316588 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" event={"ID":"f95dfaa6-6b45-4541-8827-af5e7ab0f492","Type":"ContainerStarted","Data":"d00b0145808f994b19e594f99afc3aaa4519d7d995d18bf4a52965a7c20ee55c"} Jul 15 18:51:15.317186 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:15.316919 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:51:15.317345 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:15.317331 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k5k4q" Jul 15 18:51:15.334892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:15.334845 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l9ggd" podStartSLOduration=4.402335642 podStartE2EDuration="25.334827775s" totalImagesPullingTime="20.932492133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:15.334440827 +0000 UTC m=+25.682153126" watchObservedRunningTime="2026-07-15 18:51:15.334827775 +0000 UTC m=+25.682540073" Jul 15 18:51:15.396850 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:15.396821 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:16.201069 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:16.201046 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:16.201069 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:16.201061 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:16.201243 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:16.201156 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:16.201319 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:16.201300 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:16.972601 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:16.972565 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:17.099902 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:17.099812 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:17.200273 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:17.200245 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:17.200436 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:17.200352 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:17.208772 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:17.208731 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:17.295840 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:17.295660 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:17.396609 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:17.396571 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:17.670536 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:17.670363 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:17.765980 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:17.765938 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:18.100687 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.100626 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:18.165597 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.165566 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:18.200536 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.200515 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:18.200664 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:18.200643 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:18.200701 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.200681 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:18.200769 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:18.200753 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:18.325133 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.325099 2574 generic.go:356] "Generic (PLEG): container finished" podID="4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6" containerID="50e33fca377a8d3c1e94acfacb5ae5a79c5ba01f57ff0f7d85130ede686776f1" exitCode=0 Jul 15 18:51:18.325268 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.325151 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcdnw" event={"ID":"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6","Type":"ContainerDied","Data":"50e33fca377a8d3c1e94acfacb5ae5a79c5ba01f57ff0f7d85130ede686776f1"} Jul 15 18:51:18.328627 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.328609 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 18:51:18.328999 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.328978 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" event={"ID":"e241c2dc-dedc-4a81-8e1b-6a5488efeddb","Type":"ContainerStarted","Data":"37f214caab418ed1fec7b051a1e55412f982c8a51c1e54ea788f6a4a546b8df4"} Jul 15 18:51:18.329229 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.329212 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:51:18.329323 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.329272 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:51:18.329386 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.329374 2574 scope.go:117] "RemoveContainer" containerID="904ddd3214b2302d3a562dd35476f8748fd67ca76425cc18579de0153d2774a3" Jul 15 18:51:18.346917 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.346899 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:51:18.419220 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.419197 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:18.488073 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.488042 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:18.587995 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:18.587969 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:19.200341 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.200311 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:19.200768 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:19.200437 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:19.335315 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.335290 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 18:51:19.335680 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.335660 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" event={"ID":"e241c2dc-dedc-4a81-8e1b-6a5488efeddb","Type":"ContainerStarted","Data":"0b1ea9862eb733cc064ce8377cf7fb2ff2157c9e3ba4d045c4cf0330fc16e88c"} Jul 15 18:51:19.336077 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.336055 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:51:19.350347 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.350329 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:51:19.362806 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.362770 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:19.366935 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.366900 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" podStartSLOduration=10.703508185 podStartE2EDuration="29.366885453s" totalImagesPullingTime="18.663377268s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:19.366079734 +0000 UTC m=+29.713792080" watchObservedRunningTime="2026-07-15 18:51:19.366885453 +0000 UTC m=+29.714597744" Jul 15 18:51:19.463366 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.463290 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:19.874580 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.874322 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mw4n7"] Jul 15 18:51:19.874727 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.874605 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:19.874727 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:19.874703 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:19.877702 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.877675 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9669b"] Jul 15 18:51:19.877852 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.877820 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:19.877952 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:19.877932 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:19.878369 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.878340 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z65fv"] Jul 15 18:51:19.878469 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:19.878426 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:19.878531 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:19.878502 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:20.339875 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:20.339841 2574 generic.go:356] "Generic (PLEG): container finished" podID="4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6" containerID="35c1dd990e1a02bae8b0d7e079f61aea3b035139048d625aa67f0c4b3a549574" exitCode=0 Jul 15 18:51:20.340214 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:20.339876 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcdnw" event={"ID":"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6","Type":"ContainerDied","Data":"35c1dd990e1a02bae8b0d7e079f61aea3b035139048d625aa67f0c4b3a549574"} Jul 15 18:51:20.364708 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:20.364689 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nsvqx_72394437-d6bc-4577-8e92-ce60aec0f34a/dns-node-resolver/0.log" Jul 15 18:51:20.568974 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:20.568898 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:21.125751 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:21.125719 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:21.164092 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:21.164043 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2mlqx_0673f240-f3b2-45cc-bfe2-fe7d49b5cd04/node-ca/0.log" Jul 15 18:51:21.200966 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:21.200938 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:21.201059 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:21.201032 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:21.201059 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:21.201040 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:21.201059 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:21.201058 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:21.201158 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:21.201139 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:21.201207 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:21.201192 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:21.330119 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:21.330086 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:21.425752 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:21.425719 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:21.524226 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:21.524182 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:21.626045 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:21.626013 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:21.729950 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:21.729917 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:21.987298 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:21.987217 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:22.335695 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:22.335614 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:22.345400 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:22.345373 2574 generic.go:356] "Generic (PLEG): container finished" podID="4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6" containerID="5e21778c0e48d1aa8f37c1bd015cb6316b1bc5f01ca2e11ddbf7e502875d8e6e" exitCode=0 Jul 15 18:51:22.345528 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:22.345433 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcdnw" event={"ID":"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6","Type":"ContainerDied","Data":"5e21778c0e48d1aa8f37c1bd015cb6316b1bc5f01ca2e11ddbf7e502875d8e6e"} Jul 15 18:51:22.438046 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:22.438017 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:23.188998 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:23.188953 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:23.200762 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:23.200729 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:23.200940 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:23.200849 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:23.200940 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:23.200877 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:23.200940 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:23.200869 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:23.201089 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:23.200961 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:23.201089 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:23.201057 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:23.384203 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:23.384167 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:23.714474 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:23.714435 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:23.816807 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:23.816754 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:23.876455 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:23.876406 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:23.948205 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:23.948171 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:23.948377 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:23.948217 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:23.948377 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:23.948349 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:51:23.948517 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:23.948427 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret podName:ae918cfe-ebcc-4398-b50c-dacff69aa164 nodeName:}" failed. No retries permitted until 2026-07-15 18:51:55.948409527 +0000 UTC m=+66.296121808 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret") pod "global-pull-secret-syncer-z65fv" (UID: "ae918cfe-ebcc-4398-b50c-dacff69aa164") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:51:23.948517 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:23.948357 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:51:23.948517 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:23.948495 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs podName:5ec8ecc8-2f20-485f-ab3f-738e556a7dc8 nodeName:}" failed. No retries permitted until 2026-07-15 18:51:55.948479007 +0000 UTC m=+66.296191299 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs") pod "network-metrics-daemon-9669b" (UID: "5ec8ecc8-2f20-485f-ab3f-738e556a7dc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:51:23.984823 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:23.984724 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:24.048746 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:24.048716 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6gsv\" (UniqueName: \"kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv\") pod \"network-check-target-mw4n7\" (UID: \"e79232c6-770d-4b00-80e3-43476bc0b64a\") " pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:24.048955 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:24.048925 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:51:24.048955 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:24.048954 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:51:24.049141 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:24.048968 2574 projected.go:196] Error preparing data for projected volume kube-api-access-g6gsv for pod openshift-network-diagnostics/network-check-target-mw4n7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:51:24.049141 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:24.049027 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv podName:e79232c6-770d-4b00-80e3-43476bc0b64a nodeName:}" failed. No retries permitted until 2026-07-15 18:51:56.049011299 +0000 UTC m=+66.396723578 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-g6gsv" (UniqueName: "kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv") pod "network-check-target-mw4n7" (UID: "e79232c6-770d-4b00-80e3-43476bc0b64a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:51:24.593334 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:24.593296 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:24.794950 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:24.794918 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:24.893825 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:24.893719 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:25.200899 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:25.200820 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:25.201100 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:25.200820 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:25.201100 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:25.200941 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:25.201100 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:25.201045 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:25.201100 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:25.200820 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:25.201287 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:25.201122 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:25.219596 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:25.219565 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:25.277706 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:25.277657 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:25.982699 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:25.982669 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:26.084320 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:26.084269 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:26.404098 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:26.404010 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:26.497461 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:26.497427 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:26.597669 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:26.597634 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:26.787497 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:26.787340 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:26.985721 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:26.985691 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:27.200668 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:27.200592 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:27.200828 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:27.200700 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:27.200828 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:27.200709 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:27.200828 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:27.200707 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:27.200828 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:27.200812 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:27.200966 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:27.200919 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:27.360739 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:27.360706 2574 generic.go:356] "Generic (PLEG): container finished" podID="4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6" containerID="3fbcbe2df112b1fc0db7a23e58e1fd480d8bf67a44dfe3d10d395a2f2a08605e" exitCode=0 Jul 15 18:51:27.360903 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:27.360748 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcdnw" event={"ID":"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6","Type":"ContainerDied","Data":"3fbcbe2df112b1fc0db7a23e58e1fd480d8bf67a44dfe3d10d395a2f2a08605e"} Jul 15 18:51:27.983306 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:27.983108 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:28.365910 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:28.365777 2574 generic.go:356] "Generic (PLEG): container finished" podID="4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6" containerID="fa19ba4b2cdcc46e091cfae07fe3f800b692cfad6cad9672a4b5a4ad27e49c15" exitCode=0 Jul 15 18:51:28.365910 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:28.365840 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcdnw" event={"ID":"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6","Type":"ContainerDied","Data":"fa19ba4b2cdcc46e091cfae07fe3f800b692cfad6cad9672a4b5a4ad27e49c15"} Jul 15 18:51:28.400824 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:28.400761 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:28.499663 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:28.499630 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:28.598906 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:28.598869 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:29.025540 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:29.025509 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:29.081895 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:29.081865 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:29.200300 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:29.200266 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:29.200469 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:29.200363 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:29.200469 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:29.200370 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:29.200469 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:29.200389 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:29.200563 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:29.200465 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:29.200563 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:29.200533 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:29.371415 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:29.371344 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xcdnw" event={"ID":"4c89d0f4-91c2-493b-a3a8-e0ee2c6e41e6","Type":"ContainerStarted","Data":"1f732e33db41e1c7c2cb5535455c3863d0b1c9d33dcfccb3e0f3938ab5f26ee6"} Jul 15 18:51:29.393287 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:29.393230 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xcdnw" podStartSLOduration=8.920575366 podStartE2EDuration="39.393211648s" totalImagesPullingTime="30.472636282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:29.392180948 +0000 UTC m=+39.739893247" watchObservedRunningTime="2026-07-15 18:51:29.393211648 +0000 UTC m=+39.740923949" Jul 15 18:51:29.636672 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:29.636581 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:29.687355 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:29.687328 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:29.984231 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:29.984146 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:30.182864 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:30.182834 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:30.387982 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:30.387897 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:30.779908 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:30.779801 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:31.200505 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:31.200419 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:31.200505 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:31.200470 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:31.200686 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:31.200432 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:31.200686 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:31.200532 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:31.200686 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:31.200596 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:31.200686 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:31.200670 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:31.398778 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:31.398744 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:31.588770 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:31.588691 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:31.790428 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:31.790394 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:31.981545 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:31.981455 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:32.346489 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:32.346407 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:33.200739 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:33.200707 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:33.201143 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:33.200712 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:33.201143 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:33.200825 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:33.201143 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:33.200715 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:33.201143 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:33.200886 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:33.201143 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:33.200949 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:35.200816 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:35.200769 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:35.201184 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:35.200776 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:35.201184 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:35.200886 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:35.201184 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:35.200914 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:35.201184 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:35.201012 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:35.201184 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:35.201107 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:37.200333 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:37.200301 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:37.200707 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:37.200301 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:37.200707 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:37.200412 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:37.200707 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:37.200307 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:37.200707 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:37.200477 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:37.200707 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:37.200541 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:39.200283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:39.200238 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:39.200283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:39.200284 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:39.200717 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:39.200244 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:39.200717 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:39.200362 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z65fv" podUID="ae918cfe-ebcc-4398-b50c-dacff69aa164" Jul 15 18:51:39.200717 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:39.200422 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mw4n7" podUID="e79232c6-770d-4b00-80e3-43476bc0b64a" Jul 15 18:51:39.200717 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:51:39.200508 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9669b" podUID="5ec8ecc8-2f20-485f-ab3f-738e556a7dc8" Jul 15 18:51:40.198035 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.197989 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:40.286207 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.286169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:40.389496 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.389459 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:40.539483 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.539449 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:40.933662 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.933636 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-11.ec2.internal" event="NodeReady" Jul 15 18:51:40.933860 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.933758 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 18:51:40.971938 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.971906 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68965cfc66-dz4zt"] Jul 15 18:51:40.974710 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.974693 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:40.976935 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.976901 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 18:51:40.977418 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.977404 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 18:51:40.977561 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.977546 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vb6tl\"" Jul 15 18:51:40.977606 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.977544 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 18:51:40.984112 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.984095 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 18:51:40.995315 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.995291 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68965cfc66-dz4zt"] Jul 15 18:51:40.996068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.996045 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qtljf"] Jul 15 18:51:40.999015 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.998998 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hltxx"] Jul 15 18:51:40.999129 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:40.999112 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.001949 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.001933 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.002064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.002047 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mxlws\"" Jul 15 18:51:41.002342 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.002320 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 18:51:41.002433 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.002421 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 18:51:41.004567 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.004551 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 18:51:41.004684 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.004667 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 18:51:41.005005 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.004976 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 18:51:41.005092 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.005028 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 18:51:41.005198 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.005181 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hvwrn\"" Jul 15 18:51:41.016359 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.016281 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hltxx"] Jul 15 18:51:41.016897 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.016880 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qtljf"] Jul 15 18:51:41.079027 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.078945 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-config-volume\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.079027 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.078977 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-data-volume\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.079027 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.078996 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48b8c77a-430f-4480-940e-0c6d33f5426f-trusted-ca\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.079027 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079020 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-tmp-dir\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.079351 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079127 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9rsk\" (UniqueName: \"kubernetes.io/projected/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-kube-api-access-q9rsk\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.079351 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079179 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-crio-socket\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.079351 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079208 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.079351 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079234 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48b8c77a-430f-4480-940e-0c6d33f5426f-ca-trust-extracted\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.079351 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079264 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48b8c77a-430f-4480-940e-0c6d33f5426f-bound-sa-token\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.079351 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079295 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lcq6\" (UniqueName: \"kubernetes.io/projected/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-kube-api-access-6lcq6\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.079351 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079319 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48b8c77a-430f-4480-940e-0c6d33f5426f-registry-certificates\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.079591 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079376 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-metrics-tls\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.079591 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079399 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48b8c77a-430f-4480-940e-0c6d33f5426f-installation-pull-secrets\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.079591 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079424 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.079591 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079439 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48b8c77a-430f-4480-940e-0c6d33f5426f-image-registry-private-configuration\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.079591 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079459 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48b8c77a-430f-4480-940e-0c6d33f5426f-registry-tls\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.079591 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.079481 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbmdx\" (UniqueName: \"kubernetes.io/projected/48b8c77a-430f-4480-940e-0c6d33f5426f-kube-api-access-dbmdx\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.086839 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.086814 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mdpvq"] Jul 15 18:51:41.089614 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.089601 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mdpvq" Jul 15 18:51:41.092000 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.091978 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 18:51:41.092122 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.091983 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jj7q8\"" Jul 15 18:51:41.092122 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.092008 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 18:51:41.092122 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.092056 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 18:51:41.097433 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.097414 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mdpvq"] Jul 15 18:51:41.180797 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.180756 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.180969 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.180819 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48b8c77a-430f-4480-940e-0c6d33f5426f-image-registry-private-configuration\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.180969 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.180844 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48b8c77a-430f-4480-940e-0c6d33f5426f-registry-tls\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.180969 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.180861 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbmdx\" (UniqueName: \"kubernetes.io/projected/48b8c77a-430f-4480-940e-0c6d33f5426f-kube-api-access-dbmdx\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.180969 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.180887 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-config-volume\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.180969 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.180906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-data-volume\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.180969 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.180924 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48b8c77a-430f-4480-940e-0c6d33f5426f-trusted-ca\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.180969 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.180952 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-tmp-dir\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.181307 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.180997 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd598c73-3e74-4fdb-8367-5360f9ef28fd-cert\") pod \"ingress-canary-mdpvq\" (UID: \"dd598c73-3e74-4fdb-8367-5360f9ef28fd\") " pod="openshift-ingress-canary/ingress-canary-mdpvq" Jul 15 18:51:41.181307 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181048 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9rsk\" (UniqueName: \"kubernetes.io/projected/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-kube-api-access-q9rsk\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.181307 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181096 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-crio-socket\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.181307 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181122 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.181307 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181257 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-data-volume\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.181527 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181494 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.181625 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181600 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-config-volume\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.181734 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181634 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48b8c77a-430f-4480-940e-0c6d33f5426f-ca-trust-extracted\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.181734 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181669 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48b8c77a-430f-4480-940e-0c6d33f5426f-bound-sa-token\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.181734 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181701 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lcq6\" (UniqueName: \"kubernetes.io/projected/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-kube-api-access-6lcq6\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.181734 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181726 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48b8c77a-430f-4480-940e-0c6d33f5426f-registry-certificates\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.181907 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181767 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-metrics-tls\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.181907 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181814 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48b8c77a-430f-4480-940e-0c6d33f5426f-installation-pull-secrets\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.181907 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181842 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxtl4\" (UniqueName: \"kubernetes.io/projected/dd598c73-3e74-4fdb-8367-5360f9ef28fd-kube-api-access-mxtl4\") pod \"ingress-canary-mdpvq\" (UID: \"dd598c73-3e74-4fdb-8367-5360f9ef28fd\") " pod="openshift-ingress-canary/ingress-canary-mdpvq" Jul 15 18:51:41.181907 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.181875 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-tmp-dir\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.182047 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.182005 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48b8c77a-430f-4480-940e-0c6d33f5426f-trusted-ca\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.182047 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.182027 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-crio-socket\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.182233 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.182207 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48b8c77a-430f-4480-940e-0c6d33f5426f-ca-trust-extracted\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.183057 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.183039 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48b8c77a-430f-4480-940e-0c6d33f5426f-registry-certificates\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.185655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.185631 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48b8c77a-430f-4480-940e-0c6d33f5426f-installation-pull-secrets\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.185655 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.185647 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.185759 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.185660 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48b8c77a-430f-4480-940e-0c6d33f5426f-registry-tls\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.185759 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.185649 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-metrics-tls\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.185759 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.185690 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48b8c77a-430f-4480-940e-0c6d33f5426f-image-registry-private-configuration\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.189266 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.189241 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbmdx\" (UniqueName: \"kubernetes.io/projected/48b8c77a-430f-4480-940e-0c6d33f5426f-kube-api-access-dbmdx\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.189882 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.189859 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lcq6\" (UniqueName: \"kubernetes.io/projected/b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c-kube-api-access-6lcq6\") pod \"insights-runtime-extractor-hltxx\" (UID: \"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c\") " pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.190241 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.190224 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48b8c77a-430f-4480-940e-0c6d33f5426f-bound-sa-token\") pod \"image-registry-68965cfc66-dz4zt\" (UID: \"48b8c77a-430f-4480-940e-0c6d33f5426f\") " pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.190505 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.190486 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9rsk\" (UniqueName: \"kubernetes.io/projected/788b008d-e5d4-46bb-b45d-ae0063bdd5a5-kube-api-access-q9rsk\") pod \"dns-default-qtljf\" (UID: \"788b008d-e5d4-46bb-b45d-ae0063bdd5a5\") " pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.200429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.200409 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:41.200515 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.200439 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:41.200515 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.200442 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:41.202773 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.202756 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 18:51:41.202898 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.202820 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 18:51:41.202959 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.202899 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5nvc9\"" Jul 15 18:51:41.202959 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.202936 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 18:51:41.203042 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.203027 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rz9k9\"" Jul 15 18:51:41.203534 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.203519 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 18:51:41.210727 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.210706 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:41.277683 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.277641 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:41.282544 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.282520 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd598c73-3e74-4fdb-8367-5360f9ef28fd-cert\") pod \"ingress-canary-mdpvq\" (UID: \"dd598c73-3e74-4fdb-8367-5360f9ef28fd\") " pod="openshift-ingress-canary/ingress-canary-mdpvq" Jul 15 18:51:41.282665 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.282624 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxtl4\" (UniqueName: \"kubernetes.io/projected/dd598c73-3e74-4fdb-8367-5360f9ef28fd-kube-api-access-mxtl4\") pod \"ingress-canary-mdpvq\" (UID: \"dd598c73-3e74-4fdb-8367-5360f9ef28fd\") " pod="openshift-ingress-canary/ingress-canary-mdpvq" Jul 15 18:51:41.284075 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.284052 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:41.285042 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.285022 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd598c73-3e74-4fdb-8367-5360f9ef28fd-cert\") pod \"ingress-canary-mdpvq\" (UID: \"dd598c73-3e74-4fdb-8367-5360f9ef28fd\") " pod="openshift-ingress-canary/ingress-canary-mdpvq" Jul 15 18:51:41.292662 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.290878 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxtl4\" (UniqueName: \"kubernetes.io/projected/dd598c73-3e74-4fdb-8367-5360f9ef28fd-kube-api-access-mxtl4\") pod \"ingress-canary-mdpvq\" (UID: \"dd598c73-3e74-4fdb-8367-5360f9ef28fd\") " pod="openshift-ingress-canary/ingress-canary-mdpvq" Jul 15 18:51:41.309227 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.309197 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:41.313237 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.313212 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hltxx" Jul 15 18:51:41.399385 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.399182 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mdpvq" Jul 15 18:51:41.443588 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.443544 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68965cfc66-dz4zt"] Jul 15 18:51:41.467089 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.467063 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qtljf"] Jul 15 18:51:41.469701 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.469674 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hltxx"] Jul 15 18:51:41.481814 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:51:41.481767 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07ce01b_9fbe_41c9_bb7b_e8eb58fde37c.slice/crio-d974d0b9c0d98f4d936cde409d66b2393238db55ffe487a1846a1dd553e1e978 WatchSource:0}: Error finding container d974d0b9c0d98f4d936cde409d66b2393238db55ffe487a1846a1dd553e1e978: Status 404 returned error can't find the container with id d974d0b9c0d98f4d936cde409d66b2393238db55ffe487a1846a1dd553e1e978 Jul 15 18:51:41.523394 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.523362 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:41.543480 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.543415 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mdpvq"] Jul 15 18:51:41.547239 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:51:41.547211 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd598c73_3e74_4fdb_8367_5360f9ef28fd.slice/crio-e96974243f7984f17686e8bdac0140f774d12a294ba9a7e87174d1d013bde403 WatchSource:0}: Error finding container e96974243f7984f17686e8bdac0140f774d12a294ba9a7e87174d1d013bde403: Status 404 returned error can't find the container with id e96974243f7984f17686e8bdac0140f774d12a294ba9a7e87174d1d013bde403 Jul 15 18:51:41.590979 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.590949 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:41.828834 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.828799 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-869wl"] Jul 15 18:51:41.831922 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.831905 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-869wl" Jul 15 18:51:41.835007 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.834986 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 18:51:41.835170 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.835068 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 18:51:41.835426 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.835405 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2z97d\"" Jul 15 18:51:41.845613 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.845591 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-869wl"] Jul 15 18:51:41.861915 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.861887 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:41.887308 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.887273 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlqhc\" (UniqueName: \"kubernetes.io/projected/65c16fea-c4fb-4fe2-8199-96ad95a08e38-kube-api-access-xlqhc\") pod \"downloads-68c9cbb497-869wl\" (UID: \"65c16fea-c4fb-4fe2-8199-96ad95a08e38\") " pod="openshift-console/downloads-68c9cbb497-869wl" Jul 15 18:51:41.934914 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.934878 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:41.988430 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:41.988396 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlqhc\" (UniqueName: \"kubernetes.io/projected/65c16fea-c4fb-4fe2-8199-96ad95a08e38-kube-api-access-xlqhc\") pod \"downloads-68c9cbb497-869wl\" (UID: \"65c16fea-c4fb-4fe2-8199-96ad95a08e38\") " pod="openshift-console/downloads-68c9cbb497-869wl" Jul 15 18:51:42.000559 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.000529 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlqhc\" (UniqueName: \"kubernetes.io/projected/65c16fea-c4fb-4fe2-8199-96ad95a08e38-kube-api-access-xlqhc\") pod \"downloads-68c9cbb497-869wl\" (UID: \"65c16fea-c4fb-4fe2-8199-96ad95a08e38\") " pod="openshift-console/downloads-68c9cbb497-869wl" Jul 15 18:51:42.099736 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.099640 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:42.141604 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.141320 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-869wl" Jul 15 18:51:42.345247 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.345214 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:42.358994 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.358816 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-869wl"] Jul 15 18:51:42.403152 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.403110 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hltxx" event={"ID":"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c","Type":"ContainerStarted","Data":"d974d0b9c0d98f4d936cde409d66b2393238db55ffe487a1846a1dd553e1e978"} Jul 15 18:51:42.406486 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.405655 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" event={"ID":"48b8c77a-430f-4480-940e-0c6d33f5426f","Type":"ContainerStarted","Data":"b81096474e9c2cadeff870ccee90b9dfd9d02fdc9f8afe98be523d59ffd266c2"} Jul 15 18:51:42.406486 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.405708 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" event={"ID":"48b8c77a-430f-4480-940e-0c6d33f5426f","Type":"ContainerStarted","Data":"1518850e63438928eab8b292ba85e9431656033e3612abb570338f074eef6f1e"} Jul 15 18:51:42.406486 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.406425 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:51:42.409067 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.409037 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mdpvq" event={"ID":"dd598c73-3e74-4fdb-8367-5360f9ef28fd","Type":"ContainerStarted","Data":"e96974243f7984f17686e8bdac0140f774d12a294ba9a7e87174d1d013bde403"} Jul 15 18:51:42.411297 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.411274 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qtljf" event={"ID":"788b008d-e5d4-46bb-b45d-ae0063bdd5a5","Type":"ContainerStarted","Data":"1218b9e8d29517426ab82a45fbba3738aee557fbbf30a402ecb209cb1f254a25"} Jul 15 18:51:42.414412 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.414352 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-869wl" event={"ID":"65c16fea-c4fb-4fe2-8199-96ad95a08e38","Type":"ContainerStarted","Data":"2b5739527f17361074454ef90a3179c2a086a0813e1d501c0eb33a9ac577219c"} Jul 15 18:51:42.429360 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.428152 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" podStartSLOduration=2.428132613 podStartE2EDuration="2.428132613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:42.427339876 +0000 UTC m=+52.775052176" watchObservedRunningTime="2026-07-15 18:51:42.428132613 +0000 UTC m=+52.775844908" Jul 15 18:51:42.435957 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.435871 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:42.734881 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.734768 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:42.835537 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.835492 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:42.942183 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:42.942065 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:43.294333 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:43.294282 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:43.429068 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:43.429029 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:43.530446 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:43.530410 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:43.884304 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:43.884248 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:43.982578 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:43.982545 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:44.529545 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:44.529501 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:44.590913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:44.590853 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:46.930263 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:46.930232 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7977cd69b8-c59p6"] Jul 15 18:51:46.969391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:46.969356 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7977cd69b8-c59p6"] Jul 15 18:51:46.969574 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:46.969487 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:46.972435 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:46.972390 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 18:51:46.972571 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:46.972445 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 18:51:46.972571 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:46.972393 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 18:51:46.972571 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:46.972392 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-q5h69\"" Jul 15 18:51:46.973190 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:46.973033 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 18:51:46.973190 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:46.973096 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 18:51:47.033493 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.033463 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-console-config\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.033660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.033501 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-serving-cert\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.033660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.033574 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-service-ca\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.033660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.033604 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-oauth-serving-cert\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.033660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.033648 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-oauth-config\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.033893 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.033706 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6h49\" (UniqueName: \"kubernetes.io/projected/4c05822f-6b83-4bd6-bad6-09f674600269-kube-api-access-h6h49\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.134376 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.134338 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-oauth-config\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.134539 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.134387 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6h49\" (UniqueName: \"kubernetes.io/projected/4c05822f-6b83-4bd6-bad6-09f674600269-kube-api-access-h6h49\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.134539 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.134429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-console-config\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.134539 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.134458 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-serving-cert\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.134700 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.134554 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-service-ca\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.134700 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.134589 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-oauth-serving-cert\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.135377 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.135325 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-console-config\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.135377 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.135342 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-oauth-serving-cert\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.135635 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.135610 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-service-ca\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.137363 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.137316 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-oauth-config\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.138196 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.138162 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-serving-cert\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.143199 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.143175 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6h49\" (UniqueName: \"kubernetes.io/projected/4c05822f-6b83-4bd6-bad6-09f674600269-kube-api-access-h6h49\") pod \"console-7977cd69b8-c59p6\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.281443 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.281407 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:47.913685 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:47.913655 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7977cd69b8-c59p6"] Jul 15 18:51:47.917852 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:51:47.917774 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c05822f_6b83_4bd6_bad6_09f674600269.slice/crio-73dd1ce927d7732a61031db615ca53a48bc6088da945933954907f7086c10fcd WatchSource:0}: Error finding container 73dd1ce927d7732a61031db615ca53a48bc6088da945933954907f7086c10fcd: Status 404 returned error can't find the container with id 73dd1ce927d7732a61031db615ca53a48bc6088da945933954907f7086c10fcd Jul 15 18:51:48.434703 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:48.434663 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hltxx" event={"ID":"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c","Type":"ContainerStarted","Data":"e135329a38a2dfe9edc9982f8b4a142d656eeb7bcb399ecd60c0eb908661a4e8"} Jul 15 18:51:48.436365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:48.436337 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mdpvq" event={"ID":"dd598c73-3e74-4fdb-8367-5360f9ef28fd","Type":"ContainerStarted","Data":"034bc46739ddbdd15ac832f2a86869546aa0c7ffe6cdd5dfca316541b77fbca1"} Jul 15 18:51:48.437721 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:48.437690 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7977cd69b8-c59p6" event={"ID":"4c05822f-6b83-4bd6-bad6-09f674600269","Type":"ContainerStarted","Data":"73dd1ce927d7732a61031db615ca53a48bc6088da945933954907f7086c10fcd"} Jul 15 18:51:48.439473 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:48.439448 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qtljf" event={"ID":"788b008d-e5d4-46bb-b45d-ae0063bdd5a5","Type":"ContainerStarted","Data":"b685ed5c4e25c2e04ea1a6fe3b050efb8fade3af66f27c7a224223077f8a7369"} Jul 15 18:51:48.439582 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:48.439494 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qtljf" event={"ID":"788b008d-e5d4-46bb-b45d-ae0063bdd5a5","Type":"ContainerStarted","Data":"6126ea0a3b7d13625f3545e1c4ade0fc085ecfbefeac8e6e7b9e955379b22e7e"} Jul 15 18:51:48.439644 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:48.439600 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:48.453652 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:48.453604 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mdpvq" podStartSLOduration=1.239050628 podStartE2EDuration="7.453587535s" totalImagesPullingTime="6.214536907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:48.453544569 +0000 UTC m=+58.801256868" watchObservedRunningTime="2026-07-15 18:51:48.453587535 +0000 UTC m=+58.801299828" Jul 15 18:51:48.472688 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:48.472641 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qtljf" podStartSLOduration=2.182332583 podStartE2EDuration="8.472622164s" totalImagesPullingTime="6.290289581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:51:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:48.471779296 +0000 UTC m=+58.819491597" watchObservedRunningTime="2026-07-15 18:51:48.472622164 +0000 UTC m=+58.820334464" Jul 15 18:51:51.058936 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:51.058896 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:51.147502 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:51.147441 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:51.361419 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:51.361142 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cjxwn" Jul 15 18:51:51.541260 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:51.541222 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:51.645925 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:51.645846 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:52.589574 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:52.589527 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:53.949772 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:53.949698 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:54.051830 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:54.051772 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:54.150415 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:54.150366 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:54.353489 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:54.353459 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:54.465075 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:54.464902 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7977cd69b8-c59p6" event={"ID":"4c05822f-6b83-4bd6-bad6-09f674600269","Type":"ContainerStarted","Data":"bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21"} Jul 15 18:51:54.467459 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:54.467427 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hltxx" event={"ID":"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c","Type":"ContainerStarted","Data":"de4deda6ed6e8cb15fd8d12e5bcc73e886fda6b25f09f3cd242ab7a086724ca7"} Jul 15 18:51:54.486154 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:54.486103 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7977cd69b8-c59p6" podStartSLOduration=2.712582376 podStartE2EDuration="8.486083108s" totalImagesPullingTime="5.773500732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:51:54.485503033 +0000 UTC m=+64.833215346" watchObservedRunningTime="2026-07-15 18:51:54.486083108 +0000 UTC m=+64.833795387" Jul 15 18:51:54.537637 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:54.537604 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:54.996579 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:54.996528 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:55.055764 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:55.055729 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:56.016903 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.016869 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:56.016903 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.016905 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:56.019624 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.019606 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 18:51:56.020405 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.020390 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 18:51:56.029814 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.029775 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae918cfe-ebcc-4398-b50c-dacff69aa164-original-pull-secret\") pod \"global-pull-secret-syncer-z65fv\" (UID: \"ae918cfe-ebcc-4398-b50c-dacff69aa164\") " pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:56.029918 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.029896 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec8ecc8-2f20-485f-ab3f-738e556a7dc8-metrics-certs\") pod \"network-metrics-daemon-9669b\" (UID: \"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8\") " pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:56.117419 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.117366 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6gsv\" (UniqueName: \"kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv\") pod \"network-check-target-mw4n7\" (UID: \"e79232c6-770d-4b00-80e3-43476bc0b64a\") " pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:56.120051 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.120026 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 18:51:56.130474 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.130450 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 18:51:56.144709 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.141081 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6gsv\" (UniqueName: \"kubernetes.io/projected/e79232c6-770d-4b00-80e3-43476bc0b64a-kube-api-access-g6gsv\") pod \"network-check-target-mw4n7\" (UID: \"e79232c6-770d-4b00-80e3-43476bc0b64a\") " pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:56.211982 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.211950 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5nvc9\"" Jul 15 18:51:56.217997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.217971 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rz9k9\"" Jul 15 18:51:56.219603 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.219579 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9669b" Jul 15 18:51:56.220657 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.220630 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z65fv" Jul 15 18:51:56.226773 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.226747 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:51:56.711401 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.711361 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:56.765184 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:56.765148 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:57.121897 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:57.121856 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:57.191463 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:57.191423 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:57.282618 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:57.282566 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:57.282826 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:57.282649 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:51:57.284258 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:57.284231 2574 patch_prober.go:28] interesting pod/console-7977cd69b8-c59p6 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.11:8443/health\": dial tcp 10.133.0.11:8443: connect: connection refused" start-of-body= Jul 15 18:51:57.284366 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:57.284282 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7977cd69b8-c59p6" podUID="4c05822f-6b83-4bd6-bad6-09f674600269" containerName="console" probeResult="failure" output="Get \"https://10.133.0.11:8443/health\": dial tcp 10.133.0.11:8443: connect: connection refused" Jul 15 18:51:58.446538 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:58.446509 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qtljf" Jul 15 18:51:58.885769 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:58.885725 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:51:58.939062 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:51:58.939025 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:00.171161 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:00.171126 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:01.288677 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:01.288640 2574 patch_prober.go:28] interesting pod/image-registry-68965cfc66-dz4zt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 18:52:01.289102 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:01.288698 2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" podUID="48b8c77a-430f-4480-940e-0c6d33f5426f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 18:52:03.491653 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:03.491606 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:03.550913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:03.550873 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:04.105668 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.105625 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:04.153629 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.153592 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-5jkgt"] Jul 15 18:52:04.186644 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.186605 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:04.323019 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.322982 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.326274 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.326248 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 15 18:52:04.326274 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.326270 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-chnmh\"" Jul 15 18:52:04.326489 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.326260 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 18:52:04.326489 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.326387 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 18:52:04.326595 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.326585 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 15 18:52:04.326820 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.326800 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 18:52:04.326921 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.326904 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-5jkgt"] Jul 15 18:52:04.387129 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.387032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:04.424823 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.424769 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68965cfc66-dz4zt" Jul 15 18:52:04.481189 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.481149 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/af09c313-fdac-4419-b281-c4e673280f09-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.481334 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.481226 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af09c313-fdac-4419-b281-c4e673280f09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.481334 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.481276 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af09c313-fdac-4419-b281-c4e673280f09-metrics-client-ca\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.481334 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.481301 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsfnb\" (UniqueName: \"kubernetes.io/projected/af09c313-fdac-4419-b281-c4e673280f09-kube-api-access-wsfnb\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.578187 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.578147 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:04.582092 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.582073 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsfnb\" (UniqueName: \"kubernetes.io/projected/af09c313-fdac-4419-b281-c4e673280f09-kube-api-access-wsfnb\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.582165 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.582135 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/af09c313-fdac-4419-b281-c4e673280f09-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.582298 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.582179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af09c313-fdac-4419-b281-c4e673280f09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.582369 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.582322 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af09c313-fdac-4419-b281-c4e673280f09-metrics-client-ca\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.583023 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.583002 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af09c313-fdac-4419-b281-c4e673280f09-metrics-client-ca\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.584685 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.584665 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/af09c313-fdac-4419-b281-c4e673280f09-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.584685 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.584675 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af09c313-fdac-4419-b281-c4e673280f09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.590486 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.590466 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsfnb\" (UniqueName: \"kubernetes.io/projected/af09c313-fdac-4419-b281-c4e673280f09-kube-api-access-wsfnb\") pod \"prometheus-operator-64785b6675-5jkgt\" (UID: \"af09c313-fdac-4419-b281-c4e673280f09\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.632445 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.632409 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" Jul 15 18:52:04.718283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.718250 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:04.818916 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.818861 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:04.929291 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.929253 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9669b"] Jul 15 18:52:04.932481 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:52:04.932454 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ec8ecc8_2f20_485f_ab3f_738e556a7dc8.slice/crio-857f74c620ba2d3af9ceee3647c9e3bd4cd8490b4723a28ca88376ec27dfa7d7 WatchSource:0}: Error finding container 857f74c620ba2d3af9ceee3647c9e3bd4cd8490b4723a28ca88376ec27dfa7d7: Status 404 returned error can't find the container with id 857f74c620ba2d3af9ceee3647c9e3bd4cd8490b4723a28ca88376ec27dfa7d7 Jul 15 18:52:04.950242 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:04.950207 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z65fv"] Jul 15 18:52:04.953546 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:52:04.953522 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae918cfe_ebcc_4398_b50c_dacff69aa164.slice/crio-310739e486ad5775183020dc4f8625bf5d03931c487adb7ac14cad061980db82 WatchSource:0}: Error finding container 310739e486ad5775183020dc4f8625bf5d03931c487adb7ac14cad061980db82: Status 404 returned error can't find the container with id 310739e486ad5775183020dc4f8625bf5d03931c487adb7ac14cad061980db82 Jul 15 18:52:05.175290 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.175194 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mw4n7"] Jul 15 18:52:05.178311 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.178172 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-5jkgt"] Jul 15 18:52:05.178541 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:52:05.178508 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode79232c6_770d_4b00_80e3_43476bc0b64a.slice/crio-f95184a87e944387619222f8a027dc7b99fc04b0a016d315625cdaed2bfa0c8a WatchSource:0}: Error finding container f95184a87e944387619222f8a027dc7b99fc04b0a016d315625cdaed2bfa0c8a: Status 404 returned error can't find the container with id f95184a87e944387619222f8a027dc7b99fc04b0a016d315625cdaed2bfa0c8a Jul 15 18:52:05.182120 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:52:05.182093 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf09c313_fdac_4419_b281_c4e673280f09.slice/crio-bd5131e7747b5d6250df90444e61253143bee74d909adfb5aeadf0a029402268 WatchSource:0}: Error finding container bd5131e7747b5d6250df90444e61253143bee74d909adfb5aeadf0a029402268: Status 404 returned error can't find the container with id bd5131e7747b5d6250df90444e61253143bee74d909adfb5aeadf0a029402268 Jul 15 18:52:05.507301 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.507257 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" event={"ID":"af09c313-fdac-4419-b281-c4e673280f09","Type":"ContainerStarted","Data":"bd5131e7747b5d6250df90444e61253143bee74d909adfb5aeadf0a029402268"} Jul 15 18:52:05.509515 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.509477 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z65fv" event={"ID":"ae918cfe-ebcc-4398-b50c-dacff69aa164","Type":"ContainerStarted","Data":"310739e486ad5775183020dc4f8625bf5d03931c487adb7ac14cad061980db82"} Jul 15 18:52:05.511502 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.511413 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mw4n7" event={"ID":"e79232c6-770d-4b00-80e3-43476bc0b64a","Type":"ContainerStarted","Data":"f95184a87e944387619222f8a027dc7b99fc04b0a016d315625cdaed2bfa0c8a"} Jul 15 18:52:05.513008 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.512936 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9669b" event={"ID":"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8","Type":"ContainerStarted","Data":"857f74c620ba2d3af9ceee3647c9e3bd4cd8490b4723a28ca88376ec27dfa7d7"} Jul 15 18:52:05.515262 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.515195 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-869wl" event={"ID":"65c16fea-c4fb-4fe2-8199-96ad95a08e38","Type":"ContainerStarted","Data":"ab9dbc8e9b72a75761f211977492d8d2c085a072cf8d85730e6fb9b1022e66e3"} Jul 15 18:52:05.515684 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.515652 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-869wl" Jul 15 18:52:05.519353 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.519329 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hltxx" event={"ID":"b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c","Type":"ContainerStarted","Data":"725ccb3007a33c5426efeec0c161096a1000194b3771412ae981e336b0678523"} Jul 15 18:52:05.526995 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.526964 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-869wl" Jul 15 18:52:05.532557 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.532511 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:05.535452 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.535400 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-869wl" podStartSLOduration=2.087271969 podStartE2EDuration="24.535384759s" totalImagesPullingTime="22.44811279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:52:05.5342578 +0000 UTC m=+75.881970142" watchObservedRunningTime="2026-07-15 18:52:05.535384759 +0000 UTC m=+75.883097058" Jul 15 18:52:05.555328 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.555266 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hltxx" podStartSLOduration=2.4216606609999998 podStartE2EDuration="25.555246959s" totalImagesPullingTime="23.133586298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:51:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:52:05.554377491 +0000 UTC m=+75.902089792" watchObservedRunningTime="2026-07-15 18:52:05.555246959 +0000 UTC m=+75.902959258" Jul 15 18:52:05.634643 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.634499 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:05.876657 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:05.876482 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:07.291847 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:07.290389 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:52:07.297726 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:07.297702 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:52:07.446316 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:07.446063 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:07.577621 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:07.577483 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:07.677716 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:07.677665 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:08.543008 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:08.542349 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" event={"ID":"af09c313-fdac-4419-b281-c4e673280f09","Type":"ContainerStarted","Data":"d925a656a5bdb1a9b73c0df2b753492b638467373513c274b0b3d23222fe14e0"} Jul 15 18:52:08.550298 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:08.548077 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9669b" event={"ID":"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8","Type":"ContainerStarted","Data":"9b43d2f301598dbed04cb58e705f0cbe1c40ac910fe3ba8f3cdff9d47454593f"} Jul 15 18:52:08.615214 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:08.615104 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:09.917942 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:09.917854 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:10.018856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:10.018732 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:12.128505 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.128471 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:12.186164 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.186125 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:12.567016 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.566979 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" event={"ID":"af09c313-fdac-4419-b281-c4e673280f09","Type":"ContainerStarted","Data":"1732ceb5dccef9043963624d3bdbc1e535294d447baeb2829f7af3520ad6be23"} Jul 15 18:52:12.568770 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.568737 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z65fv" event={"ID":"ae918cfe-ebcc-4398-b50c-dacff69aa164","Type":"ContainerStarted","Data":"94bad30e86d576f5897ad258ee69b0d55f36ebb46ad1a20bd9504fc102f4225b"} Jul 15 18:52:12.570537 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.570507 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mw4n7" event={"ID":"e79232c6-770d-4b00-80e3-43476bc0b64a","Type":"ContainerStarted","Data":"7a943a92deb07736dd04376e6db503e6e1f5e478dd1b72f1beefa70e5c74c8b5"} Jul 15 18:52:12.570668 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.570590 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:52:12.572677 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.572652 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9669b" event={"ID":"5ec8ecc8-2f20-485f-ab3f-738e556a7dc8","Type":"ContainerStarted","Data":"869bda7be0ad6e04a643923b9770a39d69e81b73cefbb9673f9200d7627fe695"} Jul 15 18:52:12.584586 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.584525 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-5jkgt" podStartSLOduration=5.554266262 podStartE2EDuration="8.584506396s" totalImagesPullingTime="3.030240134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:52:12.583216442 +0000 UTC m=+82.930928767" watchObservedRunningTime="2026-07-15 18:52:12.584506396 +0000 UTC m=+82.932218696" Jul 15 18:52:12.598251 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.598199 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9669b" podStartSLOduration=79.323027365 podStartE2EDuration="1m22.598178553s" totalImagesPullingTime="3.275151188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:52:12.59762134 +0000 UTC m=+82.945333659" watchObservedRunningTime="2026-07-15 18:52:12.598178553 +0000 UTC m=+82.945890854" Jul 15 18:52:12.611114 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.611055 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z65fv" podStartSLOduration=75.760813164 podStartE2EDuration="1m22.611034047s" totalImagesPullingTime="6.850220883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:52:12.61102523 +0000 UTC m=+82.958737530" watchObservedRunningTime="2026-07-15 18:52:12.611034047 +0000 UTC m=+82.958746347" Jul 15 18:52:12.626340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.626279 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mw4n7" podStartSLOduration=76.014289677 podStartE2EDuration="1m22.626260366s" totalImagesPullingTime="6.611970689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:52:12.625220525 +0000 UTC m=+82.972932825" watchObservedRunningTime="2026-07-15 18:52:12.626260366 +0000 UTC m=+82.973972666" Jul 15 18:52:12.699169 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.699126 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:12.766387 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.766343 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:12.965983 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:12.965908 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7977cd69b8-c59p6"] Jul 15 18:52:13.000130 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:13.000083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:13.102507 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:13.102452 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:13.157052 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:13.157012 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:13.541936 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:13.541897 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:13.891682 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:13.891580 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:13.946798 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:13.946751 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:14.247606 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.247519 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:14.574124 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.574081 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:14.692569 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.692533 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m"] Jul 15 18:52:14.697595 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.697565 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:14.724060 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.724029 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5twct"] Jul 15 18:52:14.724193 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.724157 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.726949 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.726929 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 15 18:52:14.727158 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.727135 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 15 18:52:14.727464 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.727450 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-75g5w\"" Jul 15 18:52:14.740952 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.740931 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m"] Jul 15 18:52:14.741031 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.741009 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.743949 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.743906 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 18:52:14.744149 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.744121 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 18:52:14.744683 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.744664 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 18:52:14.744755 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.744680 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ls8b7\"" Jul 15 18:52:14.788058 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.788034 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:14.867504 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867429 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt6bt\" (UniqueName: \"kubernetes.io/projected/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-kube-api-access-rt6bt\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.867504 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867474 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.867664 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-wtmp\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.867664 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867535 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qnvg\" (UniqueName: \"kubernetes.io/projected/33a58e62-34c0-41e9-afb1-84b6615367d0-kube-api-access-9qnvg\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.867664 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867559 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33a58e62-34c0-41e9-afb1-84b6615367d0-metrics-client-ca\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.867664 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867577 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.867664 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867633 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33a58e62-34c0-41e9-afb1-84b6615367d0-sys\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.867664 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867662 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/33a58e62-34c0-41e9-afb1-84b6615367d0-root\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.867899 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867684 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.867899 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867706 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-tls\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.867899 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867726 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.867899 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867811 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.867899 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.867836 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-textfile\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.889597 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.889572 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:14.968733 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.968693 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.968733 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.968738 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-wtmp\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.968957 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.968758 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qnvg\" (UniqueName: \"kubernetes.io/projected/33a58e62-34c0-41e9-afb1-84b6615367d0-kube-api-access-9qnvg\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.968957 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.968806 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33a58e62-34c0-41e9-afb1-84b6615367d0-metrics-client-ca\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.968957 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.968836 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.968957 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.968863 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33a58e62-34c0-41e9-afb1-84b6615367d0-sys\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.968957 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.968887 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/33a58e62-34c0-41e9-afb1-84b6615367d0-root\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.968957 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.968939 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/33a58e62-34c0-41e9-afb1-84b6615367d0-root\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.968957 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.968938 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-wtmp\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.969283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.968974 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33a58e62-34c0-41e9-afb1-84b6615367d0-sys\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.969283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.969013 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.969283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.969055 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-tls\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.969283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.969082 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.969283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.969111 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.969283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.969138 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-textfile\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.969283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.969173 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt6bt\" (UniqueName: \"kubernetes.io/projected/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-kube-api-access-rt6bt\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.969632 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.969519 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33a58e62-34c0-41e9-afb1-84b6615367d0-metrics-client-ca\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.969632 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.969575 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.969750 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.969703 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-textfile\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.969894 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.969871 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.971385 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.971363 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.971484 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.971451 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.971484 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.971473 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:14.971677 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.971660 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/33a58e62-34c0-41e9-afb1-84b6615367d0-node-exporter-tls\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.981910 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.981887 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qnvg\" (UniqueName: \"kubernetes.io/projected/33a58e62-34c0-41e9-afb1-84b6615367d0-kube-api-access-9qnvg\") pod \"node-exporter-5twct\" (UID: \"33a58e62-34c0-41e9-afb1-84b6615367d0\") " pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:14.982432 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:14.982415 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt6bt\" (UniqueName: \"kubernetes.io/projected/4f43eb57-13dd-48fa-b59c-e85ce1a54f5c-kube-api-access-rt6bt\") pod \"openshift-state-metrics-84bd68b896-brk9m\" (UID: \"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:15.032590 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:15.032555 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" Jul 15 18:52:15.049907 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:15.049883 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5twct" Jul 15 18:52:15.089478 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:52:15.089453 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a58e62_34c0_41e9_afb1_84b6615367d0.slice/crio-8f6a96f50a3d64d5f1c5c6450320b09ecf0a0608a250d562148efd3f09b7abdd WatchSource:0}: Error finding container 8f6a96f50a3d64d5f1c5c6450320b09ecf0a0608a250d562148efd3f09b7abdd: Status 404 returned error can't find the container with id 8f6a96f50a3d64d5f1c5c6450320b09ecf0a0608a250d562148efd3f09b7abdd Jul 15 18:52:15.176668 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:15.176637 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m"] Jul 15 18:52:15.179682 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:52:15.179656 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f43eb57_13dd_48fa_b59c_e85ce1a54f5c.slice/crio-2018245e98baab0d262888fe4db1a737f65837ee462f6cb1c486ef8e89fa2f9d WatchSource:0}: Error finding container 2018245e98baab0d262888fe4db1a737f65837ee462f6cb1c486ef8e89fa2f9d: Status 404 returned error can't find the container with id 2018245e98baab0d262888fe4db1a737f65837ee462f6cb1c486ef8e89fa2f9d Jul 15 18:52:15.460477 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:15.460404 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:15.522131 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:15.522090 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:15.585097 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:15.585062 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" event={"ID":"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c","Type":"ContainerStarted","Data":"115af6ac41f1e01797ef9fc6185b2b32e0578ec98a99da892e8c4ff1fa7c9070"} Jul 15 18:52:15.585293 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:15.585136 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" event={"ID":"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c","Type":"ContainerStarted","Data":"52792237011ffea991b2e0de737788ade296c9f1aa788cbfb6ae1aa65f5106bf"} Jul 15 18:52:15.585293 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:15.585166 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" event={"ID":"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c","Type":"ContainerStarted","Data":"2018245e98baab0d262888fe4db1a737f65837ee462f6cb1c486ef8e89fa2f9d"} Jul 15 18:52:15.586087 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:15.586067 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5twct" event={"ID":"33a58e62-34c0-41e9-afb1-84b6615367d0","Type":"ContainerStarted","Data":"8f6a96f50a3d64d5f1c5c6450320b09ecf0a0608a250d562148efd3f09b7abdd"} Jul 15 18:52:15.662348 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:15.662314 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:16.712121 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:16.712021 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:16.873516 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:16.873445 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:16.995019 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:16.994983 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:17.593601 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:17.593575 2574 generic.go:356] "Generic (PLEG): container finished" podID="33a58e62-34c0-41e9-afb1-84b6615367d0" containerID="3d0587e37a1c3622fe0093d0d5ebf4d68e9b30e0ec24e52c55961c840cf7c3c8" exitCode=0 Jul 15 18:52:17.593698 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:17.593638 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5twct" event={"ID":"33a58e62-34c0-41e9-afb1-84b6615367d0","Type":"ContainerDied","Data":"3d0587e37a1c3622fe0093d0d5ebf4d68e9b30e0ec24e52c55961c840cf7c3c8"} Jul 15 18:52:17.598936 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:17.598904 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:17.677615 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:17.677575 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:17.775151 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:17.775107 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:17.975558 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:17.975517 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:18.598995 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:18.598957 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" event={"ID":"4f43eb57-13dd-48fa-b59c-e85ce1a54f5c","Type":"ContainerStarted","Data":"a5ed9713859af4ac50fb9e3a51b4dc16c1115511c342ce04e931440000ec15ce"} Jul 15 18:52:18.600853 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:18.600828 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5twct" event={"ID":"33a58e62-34c0-41e9-afb1-84b6615367d0","Type":"ContainerStarted","Data":"8ac95170815b977cfea035bda656294bed57e154d320f9991d93b525bdd3572d"} Jul 15 18:52:18.600973 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:18.600872 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5twct" event={"ID":"33a58e62-34c0-41e9-afb1-84b6615367d0","Type":"ContainerStarted","Data":"9f25390fb6769f402d8228a834bfbadbaadedf22601a52c92d970adca12dc8f9"} Jul 15 18:52:18.616651 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:18.616605 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brk9m" podStartSLOduration=2.478630377 podStartE2EDuration="4.616592385s" totalImagesPullingTime="2.137962008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:52:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:52:18.615353745 +0000 UTC m=+88.963066058" watchObservedRunningTime="2026-07-15 18:52:18.616592385 +0000 UTC m=+88.964304683" Jul 15 18:52:18.633267 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:18.633225 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5twct" podStartSLOduration=3.095374038 podStartE2EDuration="4.633212406s" totalImagesPullingTime="1.45863626s" totalInitContainerRuntime="79.202108ms" isStatefulPod=true podCreationTimestamp="2026-07-15 18:52:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:52:18.631927924 +0000 UTC m=+88.979640241" watchObservedRunningTime="2026-07-15 18:52:18.633212406 +0000 UTC m=+88.980924704" Jul 15 18:52:18.948846 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:18.948752 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:19.014380 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:19.014349 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:19.287057 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:19.287023 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:19.368667 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:19.368625 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:19.447675 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:19.447645 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:19.510300 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:19.510264 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:20.948394 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:20.948355 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:24.139567 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:24.139530 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:24.196042 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:24.196008 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:25.964219 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:25.964179 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:26.053912 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:26.053880 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:28.890021 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:28.889972 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:28.954549 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:28.954520 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:30.251744 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:30.251708 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:30.306364 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:30.306327 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:30.362206 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:30.362172 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:37.990152 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:37.990105 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7977cd69b8-c59p6" podUID="4c05822f-6b83-4bd6-bad6-09f674600269" containerName="console" containerID="cri-o://bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21" gracePeriod=15 Jul 15 18:52:38.256543 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.256518 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:52:38.321022 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.320992 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-serving-cert\") pod \"4c05822f-6b83-4bd6-bad6-09f674600269\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " Jul 15 18:52:38.321022 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.321026 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-oauth-serving-cert\") pod \"4c05822f-6b83-4bd6-bad6-09f674600269\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " Jul 15 18:52:38.321248 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.321044 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-console-config\") pod \"4c05822f-6b83-4bd6-bad6-09f674600269\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " Jul 15 18:52:38.321248 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.321062 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-oauth-config\") pod \"4c05822f-6b83-4bd6-bad6-09f674600269\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " Jul 15 18:52:38.321248 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.321197 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-service-ca\") pod \"4c05822f-6b83-4bd6-bad6-09f674600269\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " Jul 15 18:52:38.321248 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.321246 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6h49\" (UniqueName: \"kubernetes.io/projected/4c05822f-6b83-4bd6-bad6-09f674600269-kube-api-access-h6h49\") pod \"4c05822f-6b83-4bd6-bad6-09f674600269\" (UID: \"4c05822f-6b83-4bd6-bad6-09f674600269\") " Jul 15 18:52:38.321567 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.321459 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4c05822f-6b83-4bd6-bad6-09f674600269" (UID: "4c05822f-6b83-4bd6-bad6-09f674600269"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:52:38.321652 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.321575 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-console-config" (OuterVolumeSpecName: "console-config") pod "4c05822f-6b83-4bd6-bad6-09f674600269" (UID: "4c05822f-6b83-4bd6-bad6-09f674600269"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:52:38.321652 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.321613 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-service-ca" (OuterVolumeSpecName: "service-ca") pod "4c05822f-6b83-4bd6-bad6-09f674600269" (UID: "4c05822f-6b83-4bd6-bad6-09f674600269"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:52:38.323260 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.323236 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4c05822f-6b83-4bd6-bad6-09f674600269" (UID: "4c05822f-6b83-4bd6-bad6-09f674600269"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 18:52:38.323365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.323316 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4c05822f-6b83-4bd6-bad6-09f674600269" (UID: "4c05822f-6b83-4bd6-bad6-09f674600269"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 18:52:38.323365 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.323347 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c05822f-6b83-4bd6-bad6-09f674600269-kube-api-access-h6h49" (OuterVolumeSpecName: "kube-api-access-h6h49") pod "4c05822f-6b83-4bd6-bad6-09f674600269" (UID: "4c05822f-6b83-4bd6-bad6-09f674600269"). InnerVolumeSpecName "kube-api-access-h6h49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:52:38.421958 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.421930 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-oauth-serving-cert\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:52:38.421958 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.421955 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-console-config\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:52:38.421958 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.421964 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-oauth-config\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:52:38.422148 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.421972 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c05822f-6b83-4bd6-bad6-09f674600269-service-ca\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:52:38.422148 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.421980 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6h49\" (UniqueName: \"kubernetes.io/projected/4c05822f-6b83-4bd6-bad6-09f674600269-kube-api-access-h6h49\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:52:38.422148 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.421988 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c05822f-6b83-4bd6-bad6-09f674600269-console-serving-cert\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:52:38.675461 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.675370 2574 generic.go:356] "Generic (PLEG): container finished" podID="4c05822f-6b83-4bd6-bad6-09f674600269" containerID="bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21" exitCode=0 Jul 15 18:52:38.675461 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.675423 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7977cd69b8-c59p6" event={"ID":"4c05822f-6b83-4bd6-bad6-09f674600269","Type":"ContainerDied","Data":"bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21"} Jul 15 18:52:38.675461 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.675429 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7977cd69b8-c59p6" Jul 15 18:52:38.675729 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.675483 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7977cd69b8-c59p6" event={"ID":"4c05822f-6b83-4bd6-bad6-09f674600269","Type":"ContainerDied","Data":"73dd1ce927d7732a61031db615ca53a48bc6088da945933954907f7086c10fcd"} Jul 15 18:52:38.675729 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.675503 2574 scope.go:117] "RemoveContainer" containerID="bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21" Jul 15 18:52:38.683688 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.683667 2574 scope.go:117] "RemoveContainer" containerID="bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21" Jul 15 18:52:38.683962 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:52:38.683943 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21\": container with ID starting with bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21 not found: ID does not exist" containerID="bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21" Jul 15 18:52:38.684017 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.683969 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21"} err="failed to get container status \"bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21\": rpc error: code = NotFound desc = could not find container \"bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21\": container with ID starting with bd4ea0f006ec3c85015daf1e47dfcc375cdc4b214237cc729edf4f5c8a366a21 not found: ID does not exist" Jul 15 18:52:38.695040 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.695013 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7977cd69b8-c59p6"] Jul 15 18:52:38.697282 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:38.697258 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7977cd69b8-c59p6"] Jul 15 18:52:40.204798 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:40.204754 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c05822f-6b83-4bd6-bad6-09f674600269" path="/var/lib/kubelet/pods/4c05822f-6b83-4bd6-bad6-09f674600269/volumes" Jul 15 18:52:43.579277 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:43.579250 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mw4n7" Jul 15 18:52:43.897114 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:43.897023 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:52:43.954076 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:52:43.954037 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:53:33.973921 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:33.973879 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:53:34.203711 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:34.203677 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:53:39.069593 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.069558 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-597fd459d4-9n88f"] Jul 15 18:53:39.069991 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.069812 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c05822f-6b83-4bd6-bad6-09f674600269" containerName="console" Jul 15 18:53:39.069991 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.069827 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c05822f-6b83-4bd6-bad6-09f674600269" containerName="console" Jul 15 18:53:39.069991 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.069871 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c05822f-6b83-4bd6-bad6-09f674600269" containerName="console" Jul 15 18:53:39.072997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.072981 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.076995 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.075626 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 18:53:39.076995 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.076181 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 18:53:39.076995 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.076388 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 18:53:39.076995 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.076561 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-qms24\"" Jul 15 18:53:39.076995 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.076724 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 18:53:39.077320 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.077001 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 18:53:39.080691 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.080669 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 18:53:39.088775 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.088755 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-597fd459d4-9n88f"] Jul 15 18:53:39.130114 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.130078 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a068110a-1711-4e92-8a2c-948a08a088d8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.130272 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.130121 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-secret-telemeter-client\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.130272 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.130182 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a068110a-1711-4e92-8a2c-948a08a088d8-serving-certs-ca-bundle\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.130272 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.130226 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.130272 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.130264 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-federate-client-tls\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.130449 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.130285 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a068110a-1711-4e92-8a2c-948a08a088d8-metrics-client-ca\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.130449 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.130300 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qr67\" (UniqueName: \"kubernetes.io/projected/a068110a-1711-4e92-8a2c-948a08a088d8-kube-api-access-9qr67\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.130449 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.130336 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-telemeter-client-tls\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.231242 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.231208 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-telemeter-client-tls\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.231417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.231246 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a068110a-1711-4e92-8a2c-948a08a088d8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.231417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.231277 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-secret-telemeter-client\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.231608 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.231444 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a068110a-1711-4e92-8a2c-948a08a088d8-serving-certs-ca-bundle\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.231608 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.231502 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.231608 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.231539 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-federate-client-tls\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.231608 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.231574 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a068110a-1711-4e92-8a2c-948a08a088d8-metrics-client-ca\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.231875 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.231613 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qr67\" (UniqueName: \"kubernetes.io/projected/a068110a-1711-4e92-8a2c-948a08a088d8-kube-api-access-9qr67\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.232242 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.232218 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a068110a-1711-4e92-8a2c-948a08a088d8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.232400 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.232375 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a068110a-1711-4e92-8a2c-948a08a088d8-metrics-client-ca\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.232502 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.232480 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a068110a-1711-4e92-8a2c-948a08a088d8-serving-certs-ca-bundle\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.233834 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.233805 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-secret-telemeter-client\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.233934 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.233836 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.233934 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.233916 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-telemeter-client-tls\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.234353 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.234336 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a068110a-1711-4e92-8a2c-948a08a088d8-federate-client-tls\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.240153 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.240133 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qr67\" (UniqueName: \"kubernetes.io/projected/a068110a-1711-4e92-8a2c-948a08a088d8-kube-api-access-9qr67\") pod \"telemeter-client-597fd459d4-9n88f\" (UID: \"a068110a-1711-4e92-8a2c-948a08a088d8\") " pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.384325 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.384261 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" Jul 15 18:53:39.517886 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.516040 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-597fd459d4-9n88f"] Jul 15 18:53:39.521582 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:53:39.521549 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda068110a_1711_4e92_8a2c_948a08a088d8.slice/crio-6a09e75aa47f0b16fc00b650469e0638abeb9ef1d47ad7f612bb7ecad3435b3f WatchSource:0}: Error finding container 6a09e75aa47f0b16fc00b650469e0638abeb9ef1d47ad7f612bb7ecad3435b3f: Status 404 returned error can't find the container with id 6a09e75aa47f0b16fc00b650469e0638abeb9ef1d47ad7f612bb7ecad3435b3f Jul 15 18:53:39.882518 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:39.882479 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" event={"ID":"a068110a-1711-4e92-8a2c-948a08a088d8","Type":"ContainerStarted","Data":"6a09e75aa47f0b16fc00b650469e0638abeb9ef1d47ad7f612bb7ecad3435b3f"} Jul 15 18:53:41.890331 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:41.890304 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" event={"ID":"a068110a-1711-4e92-8a2c-948a08a088d8","Type":"ContainerStarted","Data":"c1cc41eae0694d70aa28dd42f24b02681378882c61a0aca23276ee529b46fe9b"} Jul 15 18:53:43.899073 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:43.899040 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" event={"ID":"a068110a-1711-4e92-8a2c-948a08a088d8","Type":"ContainerStarted","Data":"fd4bd40043106620cf820b5e36755c400f6a04ff490daee572a636f1119fa9d3"} Jul 15 18:53:43.899460 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:43.899119 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" event={"ID":"a068110a-1711-4e92-8a2c-948a08a088d8","Type":"ContainerStarted","Data":"be9e57437b63f5fa2dc62f732602ca71a73440deb2920b22f4201e31405c9928"} Jul 15 18:53:43.921333 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:43.921277 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-597fd459d4-9n88f" podStartSLOduration=1.546750312 podStartE2EDuration="4.921262102s" totalImagesPullingTime="3.37451179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:53:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:53:43.920593966 +0000 UTC m=+174.268306264" watchObservedRunningTime="2026-07-15 18:53:43.921262102 +0000 UTC m=+174.268974402" Jul 15 18:53:44.630318 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.630284 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b7d7b798d-7jgtd"] Jul 15 18:53:44.633413 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.633397 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.636447 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.636423 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 18:53:44.636632 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.636619 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-q5h69\"" Jul 15 18:53:44.637707 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.637688 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 18:53:44.637822 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.637716 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 18:53:44.637822 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.637716 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 18:53:44.637822 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.637748 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 18:53:44.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.642401 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 18:53:44.645583 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.645564 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b7d7b798d-7jgtd"] Jul 15 18:53:44.687570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.687547 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:53:44.747856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.747826 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:53:44.774878 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.774857 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-oauth-serving-cert\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.774960 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.774889 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-serving-cert\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.774960 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.774914 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-oauth-config\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.775051 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.774966 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-config\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.775051 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.775005 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-trusted-ca-bundle\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.775051 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.775031 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s4rs\" (UniqueName: \"kubernetes.io/projected/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-kube-api-access-4s4rs\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.775158 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.775052 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-service-ca\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.834081 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.834041 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:53:44.875678 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.875648 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-service-ca\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.875809 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.875704 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-oauth-serving-cert\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.875809 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.875743 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-serving-cert\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.875809 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.875798 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-oauth-config\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.875972 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.875823 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-config\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.875972 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.875862 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-trusted-ca-bundle\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.875972 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.875900 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s4rs\" (UniqueName: \"kubernetes.io/projected/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-kube-api-access-4s4rs\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.876456 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.876427 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-service-ca\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.876551 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.876477 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-oauth-serving-cert\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.876551 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.876529 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-config\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.876941 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.876924 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-trusted-ca-bundle\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.878737 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.878710 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-oauth-config\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.878870 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.878855 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-serving-cert\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.885054 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.885007 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s4rs\" (UniqueName: \"kubernetes.io/projected/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-kube-api-access-4s4rs\") pod \"console-b7d7b798d-7jgtd\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:44.889852 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.889827 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:53:44.942329 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:44.942307 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:45.062867 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:45.062842 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b7d7b798d-7jgtd"] Jul 15 18:53:45.065176 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:53:45.065155 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20adbdc6_160a_4f62_a50a_f9a6c0b2f62a.slice/crio-f41d505fe08cb2db1edeb8111ef0af3c2ac3605c2abcf3eddf4d06fd983bfb7f WatchSource:0}: Error finding container f41d505fe08cb2db1edeb8111ef0af3c2ac3605c2abcf3eddf4d06fd983bfb7f: Status 404 returned error can't find the container with id f41d505fe08cb2db1edeb8111ef0af3c2ac3605c2abcf3eddf4d06fd983bfb7f Jul 15 18:53:45.428767 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:45.428729 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:53:45.494988 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:45.494957 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:53:45.907677 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:45.907645 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7d7b798d-7jgtd" event={"ID":"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a","Type":"ContainerStarted","Data":"15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b"} Jul 15 18:53:45.907846 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:45.907692 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7d7b798d-7jgtd" event={"ID":"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a","Type":"ContainerStarted","Data":"f41d505fe08cb2db1edeb8111ef0af3c2ac3605c2abcf3eddf4d06fd983bfb7f"} Jul 15 18:53:45.925629 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:45.925582 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b7d7b798d-7jgtd" podStartSLOduration=1.925565952 podStartE2EDuration="1.925565952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:53:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:53:45.924387895 +0000 UTC m=+176.272100194" watchObservedRunningTime="2026-07-15 18:53:45.925565952 +0000 UTC m=+176.273278250" Jul 15 18:53:49.215669 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:49.215627 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:53:49.316861 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:49.316823 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:53:54.943149 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:54.943121 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:54.943563 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:54.943168 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:54.947773 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:54.947752 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:55.329371 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.329336 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b7d7b798d-7jgtd"] Jul 15 18:53:55.362072 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.362042 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5db54c446c-jl5kh"] Jul 15 18:53:55.367663 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.367647 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.380082 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.380059 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db54c446c-jl5kh"] Jul 15 18:53:55.453939 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.453905 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-service-ca\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.453939 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.453942 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-console-serving-cert\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.454117 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.453963 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-console-config\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.454117 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.454032 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-oauth-serving-cert\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.454117 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.454071 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc449\" (UniqueName: \"kubernetes.io/projected/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-kube-api-access-fc449\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.454117 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.454092 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-trusted-ca-bundle\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.454243 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.454175 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-console-oauth-config\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.554744 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.554713 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-console-serving-cert\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.554867 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.554750 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-console-config\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.554867 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.554777 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-oauth-serving-cert\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.554867 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.554830 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc449\" (UniqueName: \"kubernetes.io/projected/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-kube-api-access-fc449\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.554867 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.554861 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-trusted-ca-bundle\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.555014 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.554935 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-console-oauth-config\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.555014 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.554957 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-service-ca\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.555686 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.555661 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-oauth-serving-cert\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.555814 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.555669 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-console-config\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.555814 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.555669 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-service-ca\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.556088 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.556067 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-trusted-ca-bundle\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.557366 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.557341 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-console-serving-cert\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.557447 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.557429 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-console-oauth-config\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.562759 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.562737 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc449\" (UniqueName: \"kubernetes.io/projected/8cb769cf-e4ed-4f02-8c31-49e1b636f5e0-kube-api-access-fc449\") pod \"console-5db54c446c-jl5kh\" (UID: \"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0\") " pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.675872 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.675762 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:53:55.806030 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.805998 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db54c446c-jl5kh"] Jul 15 18:53:55.809413 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:53:55.809387 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb769cf_e4ed_4f02_8c31_49e1b636f5e0.slice/crio-443ef3e1d3e256f0b6a93f40d7c09f2262cc85f67f79c0f33e4ac17117e2f084 WatchSource:0}: Error finding container 443ef3e1d3e256f0b6a93f40d7c09f2262cc85f67f79c0f33e4ac17117e2f084: Status 404 returned error can't find the container with id 443ef3e1d3e256f0b6a93f40d7c09f2262cc85f67f79c0f33e4ac17117e2f084 Jul 15 18:53:55.948512 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.948419 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db54c446c-jl5kh" event={"ID":"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0","Type":"ContainerStarted","Data":"e303189799975c557c13374398ec304ba5a225fde4cf37e67fcc30b4506e7d0c"} Jul 15 18:53:55.948512 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.948491 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db54c446c-jl5kh" event={"ID":"8cb769cf-e4ed-4f02-8c31-49e1b636f5e0","Type":"ContainerStarted","Data":"443ef3e1d3e256f0b6a93f40d7c09f2262cc85f67f79c0f33e4ac17117e2f084"} Jul 15 18:53:55.952638 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.952611 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:53:55.965912 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:53:55.965857 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5db54c446c-jl5kh" podStartSLOduration=0.965838674 podStartE2EDuration="965.838674ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:53:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:53:55.965090405 +0000 UTC m=+186.312802703" watchObservedRunningTime="2026-07-15 18:53:55.965838674 +0000 UTC m=+186.313550975" Jul 15 18:54:01.111335 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:01.111253 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:54:01.172024 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:01.171989 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:54:03.315116 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:03.315057 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:54:05.676030 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:05.675996 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:54:05.676409 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:05.676097 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:54:05.680578 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:05.680559 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:54:05.987831 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:05.987739 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5db54c446c-jl5kh" Jul 15 18:54:06.153308 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:06.153276 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:54:06.245242 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:06.245145 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:54:06.299391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:06.299361 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:54:13.925570 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:13.925532 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:54:14.022828 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:14.022779 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:54:21.970878 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:21.970817 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b7d7b798d-7jgtd" podUID="20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" containerName="console" containerID="cri-o://15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b" gracePeriod=15 Jul 15 18:54:22.203371 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.203349 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:54:22.261103 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.261073 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4s4rs\" (UniqueName: \"kubernetes.io/projected/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-kube-api-access-4s4rs\") pod \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " Jul 15 18:54:22.261246 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.261115 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-oauth-serving-cert\") pod \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " Jul 15 18:54:22.261246 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.261133 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-serving-cert\") pod \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " Jul 15 18:54:22.261246 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.261175 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-oauth-config\") pod \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " Jul 15 18:54:22.261246 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.261193 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-trusted-ca-bundle\") pod \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " Jul 15 18:54:22.261246 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.261213 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-service-ca\") pod \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " Jul 15 18:54:22.261516 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.261267 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-config\") pod \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\" (UID: \"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a\") " Jul 15 18:54:22.261622 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.261595 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" (UID: "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:54:22.261720 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.261618 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-service-ca" (OuterVolumeSpecName: "service-ca") pod "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" (UID: "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:54:22.261720 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.261664 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-config" (OuterVolumeSpecName: "console-config") pod "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" (UID: "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:54:22.261865 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.261842 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" (UID: "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:54:22.263361 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.263334 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" (UID: "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 18:54:22.263634 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.263604 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" (UID: "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 18:54:22.263634 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.263614 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-kube-api-access-4s4rs" (OuterVolumeSpecName: "kube-api-access-4s4rs") pod "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" (UID: "20adbdc6-160a-4f62-a50a-f9a6c0b2f62a"). InnerVolumeSpecName "kube-api-access-4s4rs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:54:22.362439 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.362410 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-config\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:54:22.362439 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.362435 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s4rs\" (UniqueName: \"kubernetes.io/projected/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-kube-api-access-4s4rs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:54:22.362595 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.362446 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-oauth-serving-cert\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:54:22.362595 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.362455 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-serving-cert\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:54:22.362595 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.362464 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-console-oauth-config\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:54:22.362595 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.362472 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-trusted-ca-bundle\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:54:22.362595 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:22.362480 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a-service-ca\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:54:23.042431 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:23.042391 2574 generic.go:356] "Generic (PLEG): container finished" podID="20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" containerID="15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b" exitCode=0 Jul 15 18:54:23.042905 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:23.042449 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7d7b798d-7jgtd" Jul 15 18:54:23.042905 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:23.042452 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7d7b798d-7jgtd" event={"ID":"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a","Type":"ContainerDied","Data":"15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b"} Jul 15 18:54:23.042905 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:23.042604 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7d7b798d-7jgtd" event={"ID":"20adbdc6-160a-4f62-a50a-f9a6c0b2f62a","Type":"ContainerDied","Data":"f41d505fe08cb2db1edeb8111ef0af3c2ac3605c2abcf3eddf4d06fd983bfb7f"} Jul 15 18:54:23.042905 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:23.042636 2574 scope.go:117] "RemoveContainer" containerID="15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b" Jul 15 18:54:23.050912 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:23.050897 2574 scope.go:117] "RemoveContainer" containerID="15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b" Jul 15 18:54:23.051163 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:54:23.051144 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b\": container with ID starting with 15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b not found: ID does not exist" containerID="15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b" Jul 15 18:54:23.051219 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:23.051174 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b"} err="failed to get container status \"15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b\": rpc error: code = NotFound desc = could not find container \"15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b\": container with ID starting with 15267cc5df2902fc55f0493e82e28f2cce41fcba0fc47e59d7ee7a38a3dfcb0b not found: ID does not exist" Jul 15 18:54:23.062050 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:23.062028 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b7d7b798d-7jgtd"] Jul 15 18:54:23.068398 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:23.068376 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b7d7b798d-7jgtd"] Jul 15 18:54:24.205228 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:24.205193 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" path="/var/lib/kubelet/pods/20adbdc6-160a-4f62-a50a-f9a6c0b2f62a/volumes" Jul 15 18:54:35.806942 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:54:35.806902 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:04.088886 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:04.088837 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:04.191040 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:04.191002 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:05.142254 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.142220 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z"] Jul 15 18:55:05.142603 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.142468 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" containerName="console" Jul 15 18:55:05.142603 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.142478 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" containerName="console" Jul 15 18:55:05.142603 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.142523 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="20adbdc6-160a-4f62-a50a-f9a6c0b2f62a" containerName="console" Jul 15 18:55:05.146691 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.146670 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:05.149366 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.149346 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 18:55:05.149467 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.149439 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 18:55:05.150204 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.150187 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8xgv5\"" Jul 15 18:55:05.155868 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.155848 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z"] Jul 15 18:55:05.270868 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.270834 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:05.271031 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.270888 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkpnk\" (UniqueName: \"kubernetes.io/projected/1f31473d-b556-4f90-a6ff-bb95dd188be9-kube-api-access-lkpnk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:05.271031 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.270971 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:05.372013 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.371973 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:05.372177 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.372027 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkpnk\" (UniqueName: \"kubernetes.io/projected/1f31473d-b556-4f90-a6ff-bb95dd188be9-kube-api-access-lkpnk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:05.372177 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.372068 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:05.372389 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.372370 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:05.372425 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.372378 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:05.381535 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.381512 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkpnk\" (UniqueName: \"kubernetes.io/projected/1f31473d-b556-4f90-a6ff-bb95dd188be9-kube-api-access-lkpnk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:05.456563 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.456497 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:05.580084 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:05.580060 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z"] Jul 15 18:55:05.582553 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:55:05.582526 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f31473d_b556_4f90_a6ff_bb95dd188be9.slice/crio-fc2a821a41d0aa17f974ab028fe1c06deaa03b1164f99d8b4db51784acdb59d7 WatchSource:0}: Error finding container fc2a821a41d0aa17f974ab028fe1c06deaa03b1164f99d8b4db51784acdb59d7: Status 404 returned error can't find the container with id fc2a821a41d0aa17f974ab028fe1c06deaa03b1164f99d8b4db51784acdb59d7 Jul 15 18:55:06.189299 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:06.189265 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" event={"ID":"1f31473d-b556-4f90-a6ff-bb95dd188be9","Type":"ContainerStarted","Data":"fc2a821a41d0aa17f974ab028fe1c06deaa03b1164f99d8b4db51784acdb59d7"} Jul 15 18:55:13.216566 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:13.216475 2574 generic.go:356] "Generic (PLEG): container finished" podID="1f31473d-b556-4f90-a6ff-bb95dd188be9" containerID="19cdda0a132b7ebede476ff380f942335e84e531c84644a8d4da286f602ff343" exitCode=0 Jul 15 18:55:13.216566 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:13.216547 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" event={"ID":"1f31473d-b556-4f90-a6ff-bb95dd188be9","Type":"ContainerDied","Data":"19cdda0a132b7ebede476ff380f942335e84e531c84644a8d4da286f602ff343"} Jul 15 18:55:16.228770 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:16.228738 2574 generic.go:356] "Generic (PLEG): container finished" podID="1f31473d-b556-4f90-a6ff-bb95dd188be9" containerID="d636f615943b902d1e659e4bc77343a208d68ec0b5c9a2abcfbab02e98e02f2c" exitCode=0 Jul 15 18:55:16.229143 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:16.228825 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" event={"ID":"1f31473d-b556-4f90-a6ff-bb95dd188be9","Type":"ContainerDied","Data":"d636f615943b902d1e659e4bc77343a208d68ec0b5c9a2abcfbab02e98e02f2c"} Jul 15 18:55:24.261684 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:24.261647 2574 generic.go:356] "Generic (PLEG): container finished" podID="1f31473d-b556-4f90-a6ff-bb95dd188be9" containerID="fb6d6c48975cf2c9f2fefde3145b93f5a2f3b8fad286ac1b2764aaa93257855d" exitCode=0 Jul 15 18:55:24.262062 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:24.261710 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" event={"ID":"1f31473d-b556-4f90-a6ff-bb95dd188be9","Type":"ContainerDied","Data":"fb6d6c48975cf2c9f2fefde3145b93f5a2f3b8fad286ac1b2764aaa93257855d"} Jul 15 18:55:25.385236 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:25.385210 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:25.432288 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:25.432261 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-util\") pod \"1f31473d-b556-4f90-a6ff-bb95dd188be9\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " Jul 15 18:55:25.432427 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:25.432295 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-bundle\") pod \"1f31473d-b556-4f90-a6ff-bb95dd188be9\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " Jul 15 18:55:25.432427 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:25.432346 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkpnk\" (UniqueName: \"kubernetes.io/projected/1f31473d-b556-4f90-a6ff-bb95dd188be9-kube-api-access-lkpnk\") pod \"1f31473d-b556-4f90-a6ff-bb95dd188be9\" (UID: \"1f31473d-b556-4f90-a6ff-bb95dd188be9\") " Jul 15 18:55:25.432978 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:25.432949 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-bundle" (OuterVolumeSpecName: "bundle") pod "1f31473d-b556-4f90-a6ff-bb95dd188be9" (UID: "1f31473d-b556-4f90-a6ff-bb95dd188be9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:55:25.434449 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:25.434426 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f31473d-b556-4f90-a6ff-bb95dd188be9-kube-api-access-lkpnk" (OuterVolumeSpecName: "kube-api-access-lkpnk") pod "1f31473d-b556-4f90-a6ff-bb95dd188be9" (UID: "1f31473d-b556-4f90-a6ff-bb95dd188be9"). InnerVolumeSpecName "kube-api-access-lkpnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:55:25.436454 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:25.436433 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-util" (OuterVolumeSpecName: "util") pod "1f31473d-b556-4f90-a6ff-bb95dd188be9" (UID: "1f31473d-b556-4f90-a6ff-bb95dd188be9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:55:25.533278 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:25.533211 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-util\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:55:25.533278 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:25.533234 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f31473d-b556-4f90-a6ff-bb95dd188be9-bundle\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:55:25.533278 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:25.533243 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkpnk\" (UniqueName: \"kubernetes.io/projected/1f31473d-b556-4f90-a6ff-bb95dd188be9-kube-api-access-lkpnk\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:55:26.269641 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:26.269611 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" event={"ID":"1f31473d-b556-4f90-a6ff-bb95dd188be9","Type":"ContainerDied","Data":"fc2a821a41d0aa17f974ab028fe1c06deaa03b1164f99d8b4db51784acdb59d7"} Jul 15 18:55:26.269767 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:26.269658 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc2a821a41d0aa17f974ab028fe1c06deaa03b1164f99d8b4db51784acdb59d7" Jul 15 18:55:26.269767 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:26.269657 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408j78z" Jul 15 18:55:31.537743 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:31.537703 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:31.942185 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:31.942086 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:33.738652 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.738618 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv"] Jul 15 18:55:33.739034 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.738929 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f31473d-b556-4f90-a6ff-bb95dd188be9" containerName="util" Jul 15 18:55:33.739034 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.738943 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f31473d-b556-4f90-a6ff-bb95dd188be9" containerName="util" Jul 15 18:55:33.739034 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.738959 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f31473d-b556-4f90-a6ff-bb95dd188be9" containerName="pull" Jul 15 18:55:33.739034 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.738965 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f31473d-b556-4f90-a6ff-bb95dd188be9" containerName="pull" Jul 15 18:55:33.739034 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.738974 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f31473d-b556-4f90-a6ff-bb95dd188be9" containerName="extract" Jul 15 18:55:33.739034 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.738980 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f31473d-b556-4f90-a6ff-bb95dd188be9" containerName="extract" Jul 15 18:55:33.739034 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.739025 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f31473d-b556-4f90-a6ff-bb95dd188be9" containerName="extract" Jul 15 18:55:33.742280 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.742266 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv" Jul 15 18:55:33.744931 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.744907 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:55:33.745053 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.744957 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lklcj\"" Jul 15 18:55:33.745053 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.745031 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 15 18:55:33.757767 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.757750 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv"] Jul 15 18:55:33.787817 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.787772 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77420224-8369-434b-9c3b-5ebf943915f1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5qjhv\" (UID: \"77420224-8369-434b-9c3b-5ebf943915f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv" Jul 15 18:55:33.787907 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.787840 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-647t9\" (UniqueName: \"kubernetes.io/projected/77420224-8369-434b-9c3b-5ebf943915f1-kube-api-access-647t9\") pod \"cert-manager-operator-controller-manager-55c444bf86-5qjhv\" (UID: \"77420224-8369-434b-9c3b-5ebf943915f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv" Jul 15 18:55:33.888827 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.888763 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77420224-8369-434b-9c3b-5ebf943915f1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5qjhv\" (UID: \"77420224-8369-434b-9c3b-5ebf943915f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv" Jul 15 18:55:33.889021 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.888848 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-647t9\" (UniqueName: \"kubernetes.io/projected/77420224-8369-434b-9c3b-5ebf943915f1-kube-api-access-647t9\") pod \"cert-manager-operator-controller-manager-55c444bf86-5qjhv\" (UID: \"77420224-8369-434b-9c3b-5ebf943915f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv" Jul 15 18:55:33.889217 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.889196 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77420224-8369-434b-9c3b-5ebf943915f1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5qjhv\" (UID: \"77420224-8369-434b-9c3b-5ebf943915f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv" Jul 15 18:55:33.898117 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:33.898087 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-647t9\" (UniqueName: \"kubernetes.io/projected/77420224-8369-434b-9c3b-5ebf943915f1-kube-api-access-647t9\") pod \"cert-manager-operator-controller-manager-55c444bf86-5qjhv\" (UID: \"77420224-8369-434b-9c3b-5ebf943915f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv" Jul 15 18:55:34.050885 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:34.050857 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv" Jul 15 18:55:34.192639 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:34.192613 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv"] Jul 15 18:55:34.193879 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:55:34.193848 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77420224_8369_434b_9c3b_5ebf943915f1.slice/crio-cd054b244bd40ded505fc1cc6202f1dbe26f5cb187d60ede9006b73f4746f428 WatchSource:0}: Error finding container cd054b244bd40ded505fc1cc6202f1dbe26f5cb187d60ede9006b73f4746f428: Status 404 returned error can't find the container with id cd054b244bd40ded505fc1cc6202f1dbe26f5cb187d60ede9006b73f4746f428 Jul 15 18:55:34.297190 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:34.297155 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv" event={"ID":"77420224-8369-434b-9c3b-5ebf943915f1","Type":"ContainerStarted","Data":"cd054b244bd40ded505fc1cc6202f1dbe26f5cb187d60ede9006b73f4746f428"} Jul 15 18:55:35.531258 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:35.531199 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:36.306893 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:36.306861 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv" event={"ID":"77420224-8369-434b-9c3b-5ebf943915f1","Type":"ContainerStarted","Data":"c77977c5ed81fda09782272d0dd462a4d1535b9f02be2677d704f9a7b4545128"} Jul 15 18:55:36.355444 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:36.355382 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5qjhv" podStartSLOduration=1.763842698 podStartE2EDuration="3.35536042s" totalImagesPullingTime="1.591517722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:55:36.353219821 +0000 UTC m=+286.700932120" watchObservedRunningTime="2026-07-15 18:55:36.35536042 +0000 UTC m=+286.703072720" Jul 15 18:55:37.083874 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.083836 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:37.438533 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.438444 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5"] Jul 15 18:55:37.441818 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.441774 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:37.444905 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.444882 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 18:55:37.445005 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.444943 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8xgv5\"" Jul 15 18:55:37.445676 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.445660 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 18:55:37.465007 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.464984 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5"] Jul 15 18:55:37.514120 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.514094 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:37.514245 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.514131 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:37.514245 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.514167 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94p6l\" (UniqueName: \"kubernetes.io/projected/6b77841e-5ed0-401a-bdea-5397aa401fe2-kube-api-access-94p6l\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:37.606551 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.606515 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:37.615482 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.615457 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:37.615604 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.615491 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:37.615604 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.615533 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94p6l\" (UniqueName: \"kubernetes.io/projected/6b77841e-5ed0-401a-bdea-5397aa401fe2-kube-api-access-94p6l\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:37.615914 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.615895 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:37.615993 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.615939 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:37.632329 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.632307 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94p6l\" (UniqueName: \"kubernetes.io/projected/6b77841e-5ed0-401a-bdea-5397aa401fe2-kube-api-access-94p6l\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:37.707575 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.707508 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:37.751116 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.751091 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:37.800834 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.800804 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9hdgb"] Jul 15 18:55:37.804228 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.804207 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" Jul 15 18:55:37.807585 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.807559 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 18:55:37.807700 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.807584 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 18:55:37.817209 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.817182 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9hdgb"] Jul 15 18:55:37.898629 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.898606 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5"] Jul 15 18:55:37.901412 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:55:37.901372 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b77841e_5ed0_401a_bdea_5397aa401fe2.slice/crio-344c5fff59a100947b28390814ef46d287bcfda3d3818aa72acbf4285321622d WatchSource:0}: Error finding container 344c5fff59a100947b28390814ef46d287bcfda3d3818aa72acbf4285321622d: Status 404 returned error can't find the container with id 344c5fff59a100947b28390814ef46d287bcfda3d3818aa72acbf4285321622d Jul 15 18:55:37.917744 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.917718 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e78f2b4a-4868-47ee-9e70-673718692877-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9hdgb\" (UID: \"e78f2b4a-4868-47ee-9e70-673718692877\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" Jul 15 18:55:37.917891 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:37.917870 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwxfd\" (UniqueName: \"kubernetes.io/projected/e78f2b4a-4868-47ee-9e70-673718692877-kube-api-access-wwxfd\") pod \"cert-manager-webhook-6486ffd997-9hdgb\" (UID: \"e78f2b4a-4868-47ee-9e70-673718692877\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" Jul 15 18:55:38.018880 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:38.018852 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e78f2b4a-4868-47ee-9e70-673718692877-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9hdgb\" (UID: \"e78f2b4a-4868-47ee-9e70-673718692877\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" Jul 15 18:55:38.018979 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:38.018925 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwxfd\" (UniqueName: \"kubernetes.io/projected/e78f2b4a-4868-47ee-9e70-673718692877-kube-api-access-wwxfd\") pod \"cert-manager-webhook-6486ffd997-9hdgb\" (UID: \"e78f2b4a-4868-47ee-9e70-673718692877\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" Jul 15 18:55:38.028810 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:38.028771 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwxfd\" (UniqueName: \"kubernetes.io/projected/e78f2b4a-4868-47ee-9e70-673718692877-kube-api-access-wwxfd\") pod \"cert-manager-webhook-6486ffd997-9hdgb\" (UID: \"e78f2b4a-4868-47ee-9e70-673718692877\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" Jul 15 18:55:38.031524 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:38.031506 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e78f2b4a-4868-47ee-9e70-673718692877-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9hdgb\" (UID: \"e78f2b4a-4868-47ee-9e70-673718692877\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" Jul 15 18:55:38.128465 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:38.128439 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" Jul 15 18:55:38.288945 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:38.288918 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9hdgb"] Jul 15 18:55:38.292137 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:55:38.292114 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode78f2b4a_4868_47ee_9e70_673718692877.slice/crio-e1be66876d3911ce5fb9c937dc0e13ee731850c6c0fd2bd2ef26071c0cddac76 WatchSource:0}: Error finding container e1be66876d3911ce5fb9c937dc0e13ee731850c6c0fd2bd2ef26071c0cddac76: Status 404 returned error can't find the container with id e1be66876d3911ce5fb9c937dc0e13ee731850c6c0fd2bd2ef26071c0cddac76 Jul 15 18:55:38.316759 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:38.316726 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" event={"ID":"e78f2b4a-4868-47ee-9e70-673718692877","Type":"ContainerStarted","Data":"e1be66876d3911ce5fb9c937dc0e13ee731850c6c0fd2bd2ef26071c0cddac76"} Jul 15 18:55:38.318103 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:38.318085 2574 generic.go:356] "Generic (PLEG): container finished" podID="6b77841e-5ed0-401a-bdea-5397aa401fe2" containerID="2d6ce4761b0acfaa48c0e6a1c72c56860adcdabc002514307e272e62f084feed" exitCode=0 Jul 15 18:55:38.318163 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:38.318110 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" event={"ID":"6b77841e-5ed0-401a-bdea-5397aa401fe2","Type":"ContainerDied","Data":"2d6ce4761b0acfaa48c0e6a1c72c56860adcdabc002514307e272e62f084feed"} Jul 15 18:55:38.318163 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:38.318151 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" event={"ID":"6b77841e-5ed0-401a-bdea-5397aa401fe2","Type":"ContainerStarted","Data":"344c5fff59a100947b28390814ef46d287bcfda3d3818aa72acbf4285321622d"} Jul 15 18:55:40.009854 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:40.009755 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:40.212019 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:40.211970 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:41.332073 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:41.332024 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:41.393399 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:41.393347 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:42.110538 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:42.110499 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:42.336340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:42.336306 2574 generic.go:356] "Generic (PLEG): container finished" podID="6b77841e-5ed0-401a-bdea-5397aa401fe2" containerID="b0d3c39d940c7fe8db1b45e7b8f3208fd98faa74d35835862ef0ceb859a38755" exitCode=0 Jul 15 18:55:42.336770 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:42.336390 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" event={"ID":"6b77841e-5ed0-401a-bdea-5397aa401fe2","Type":"ContainerDied","Data":"b0d3c39d940c7fe8db1b45e7b8f3208fd98faa74d35835862ef0ceb859a38755"} Jul 15 18:55:42.337870 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:42.337847 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" event={"ID":"e78f2b4a-4868-47ee-9e70-673718692877","Type":"ContainerStarted","Data":"7a31ca8f7f5efa7b2c0514ab5594a5d02b3b9999d1dbaf7de02f68554065f85f"} Jul 15 18:55:42.337991 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:42.337978 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" Jul 15 18:55:42.393423 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:42.393380 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" podStartSLOduration=2.048853063 podStartE2EDuration="5.393366094s" totalImagesPullingTime="3.344513031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:55:42.390522303 +0000 UTC m=+292.738234602" watchObservedRunningTime="2026-07-15 18:55:42.393366094 +0000 UTC m=+292.741078393" Jul 15 18:55:43.344095 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:43.344059 2574 generic.go:356] "Generic (PLEG): container finished" podID="6b77841e-5ed0-401a-bdea-5397aa401fe2" containerID="91ce12f14435b34b74b1b260ea56590fb2e9d6c1af6c14bfcc862d0442bdd0eb" exitCode=0 Jul 15 18:55:43.344495 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:43.344150 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" event={"ID":"6b77841e-5ed0-401a-bdea-5397aa401fe2","Type":"ContainerDied","Data":"91ce12f14435b34b74b1b260ea56590fb2e9d6c1af6c14bfcc862d0442bdd0eb"} Jul 15 18:55:44.469192 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:44.469167 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:44.565819 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:44.565770 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-bundle\") pod \"6b77841e-5ed0-401a-bdea-5397aa401fe2\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " Jul 15 18:55:44.565953 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:44.565836 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-util\") pod \"6b77841e-5ed0-401a-bdea-5397aa401fe2\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " Jul 15 18:55:44.565953 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:44.565883 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94p6l\" (UniqueName: \"kubernetes.io/projected/6b77841e-5ed0-401a-bdea-5397aa401fe2-kube-api-access-94p6l\") pod \"6b77841e-5ed0-401a-bdea-5397aa401fe2\" (UID: \"6b77841e-5ed0-401a-bdea-5397aa401fe2\") " Jul 15 18:55:44.566197 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:44.566177 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-bundle" (OuterVolumeSpecName: "bundle") pod "6b77841e-5ed0-401a-bdea-5397aa401fe2" (UID: "6b77841e-5ed0-401a-bdea-5397aa401fe2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:55:44.567890 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:44.567869 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b77841e-5ed0-401a-bdea-5397aa401fe2-kube-api-access-94p6l" (OuterVolumeSpecName: "kube-api-access-94p6l") pod "6b77841e-5ed0-401a-bdea-5397aa401fe2" (UID: "6b77841e-5ed0-401a-bdea-5397aa401fe2"). InnerVolumeSpecName "kube-api-access-94p6l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:55:44.570775 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:44.570739 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-util" (OuterVolumeSpecName: "util") pod "6b77841e-5ed0-401a-bdea-5397aa401fe2" (UID: "6b77841e-5ed0-401a-bdea-5397aa401fe2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:55:44.667352 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:44.667292 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-bundle\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:55:44.667352 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:44.667315 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b77841e-5ed0-401a-bdea-5397aa401fe2-util\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:55:44.667352 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:44.667324 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94p6l\" (UniqueName: \"kubernetes.io/projected/6b77841e-5ed0-401a-bdea-5397aa401fe2-kube-api-access-94p6l\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:55:45.354001 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.353972 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" Jul 15 18:55:45.354159 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.353968 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8kwt5" event={"ID":"6b77841e-5ed0-401a-bdea-5397aa401fe2","Type":"ContainerDied","Data":"344c5fff59a100947b28390814ef46d287bcfda3d3818aa72acbf4285321622d"} Jul 15 18:55:45.354159 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.354090 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="344c5fff59a100947b28390814ef46d287bcfda3d3818aa72acbf4285321622d" Jul 15 18:55:45.897350 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.897318 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7"] Jul 15 18:55:45.897696 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.897599 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b77841e-5ed0-401a-bdea-5397aa401fe2" containerName="util" Jul 15 18:55:45.897696 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.897610 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b77841e-5ed0-401a-bdea-5397aa401fe2" containerName="util" Jul 15 18:55:45.897696 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.897628 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b77841e-5ed0-401a-bdea-5397aa401fe2" containerName="extract" Jul 15 18:55:45.897696 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.897636 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b77841e-5ed0-401a-bdea-5397aa401fe2" containerName="extract" Jul 15 18:55:45.897696 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.897650 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b77841e-5ed0-401a-bdea-5397aa401fe2" containerName="pull" Jul 15 18:55:45.897696 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.897656 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b77841e-5ed0-401a-bdea-5397aa401fe2" containerName="pull" Jul 15 18:55:45.897897 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.897702 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b77841e-5ed0-401a-bdea-5397aa401fe2" containerName="extract" Jul 15 18:55:45.901406 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.901391 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7" Jul 15 18:55:45.905559 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.905531 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-zdbvq\"" Jul 15 18:55:45.920527 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.916334 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7"] Jul 15 18:55:45.977031 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.977005 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhwjn\" (UniqueName: \"kubernetes.io/projected/379c9ef6-9662-42a8-8262-ecf3c5e8527a-kube-api-access-lhwjn\") pod \"cert-manager-cainjector-559cdd8c87-d9wt7\" (UID: \"379c9ef6-9662-42a8-8262-ecf3c5e8527a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7" Jul 15 18:55:45.977031 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:45.977031 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/379c9ef6-9662-42a8-8262-ecf3c5e8527a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d9wt7\" (UID: \"379c9ef6-9662-42a8-8262-ecf3c5e8527a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7" Jul 15 18:55:46.077913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:46.077877 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhwjn\" (UniqueName: \"kubernetes.io/projected/379c9ef6-9662-42a8-8262-ecf3c5e8527a-kube-api-access-lhwjn\") pod \"cert-manager-cainjector-559cdd8c87-d9wt7\" (UID: \"379c9ef6-9662-42a8-8262-ecf3c5e8527a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7" Jul 15 18:55:46.077913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:46.077916 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/379c9ef6-9662-42a8-8262-ecf3c5e8527a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d9wt7\" (UID: \"379c9ef6-9662-42a8-8262-ecf3c5e8527a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7" Jul 15 18:55:46.087156 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:46.087125 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/379c9ef6-9662-42a8-8262-ecf3c5e8527a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d9wt7\" (UID: \"379c9ef6-9662-42a8-8262-ecf3c5e8527a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7" Jul 15 18:55:46.087265 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:46.087199 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhwjn\" (UniqueName: \"kubernetes.io/projected/379c9ef6-9662-42a8-8262-ecf3c5e8527a-kube-api-access-lhwjn\") pod \"cert-manager-cainjector-559cdd8c87-d9wt7\" (UID: \"379c9ef6-9662-42a8-8262-ecf3c5e8527a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7" Jul 15 18:55:46.210547 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:46.210480 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7" Jul 15 18:55:46.343398 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:46.343376 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7"] Jul 15 18:55:46.345464 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:55:46.345430 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod379c9ef6_9662_42a8_8262_ecf3c5e8527a.slice/crio-406e76cd7744346125f3d1afe29c0190dd3c09a149c65269e549fd805b5ebd8c WatchSource:0}: Error finding container 406e76cd7744346125f3d1afe29c0190dd3c09a149c65269e549fd805b5ebd8c: Status 404 returned error can't find the container with id 406e76cd7744346125f3d1afe29c0190dd3c09a149c65269e549fd805b5ebd8c Jul 15 18:55:46.359231 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:46.359208 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7" event={"ID":"379c9ef6-9662-42a8-8262-ecf3c5e8527a","Type":"ContainerStarted","Data":"406e76cd7744346125f3d1afe29c0190dd3c09a149c65269e549fd805b5ebd8c"} Jul 15 18:55:47.364414 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:47.364381 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7" event={"ID":"379c9ef6-9662-42a8-8262-ecf3c5e8527a","Type":"ContainerStarted","Data":"c998b84fa7241b7e0769235106c4c9af9f0de7fd4b587a7c2ae8e61c89808020"} Jul 15 18:55:47.386409 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:47.386361 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d9wt7" podStartSLOduration=2.386347403 podStartE2EDuration="2.386347403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:55:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:55:47.385600342 +0000 UTC m=+297.733312641" watchObservedRunningTime="2026-07-15 18:55:47.386347403 +0000 UTC m=+297.734059702" Jul 15 18:55:48.025861 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:48.025828 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:48.104352 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:48.104315 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:48.346470 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:48.346383 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9hdgb" Jul 15 18:55:50.083199 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:50.083169 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 18:55:50.084125 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:50.084103 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 18:55:50.087073 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:50.087050 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 18:55:50.563248 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:50.563214 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:50.713862 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:50.713824 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:56.692185 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:56.692144 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:56.788600 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:56.788568 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:58.197426 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.197388 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc"] Jul 15 18:55:58.201671 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.201643 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:55:58.204026 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.204007 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 18:55:58.204873 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.204852 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8xgv5\"" Jul 15 18:55:58.204990 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.204858 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 18:55:58.209405 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.209384 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc"] Jul 15 18:55:58.261862 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.261836 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:55:58.261997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.261878 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5pbn\" (UniqueName: \"kubernetes.io/projected/af9e5f62-ece3-4bf9-98df-e2fd243224e1-kube-api-access-r5pbn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:55:58.261997 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.261957 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:55:58.362548 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.362522 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:55:58.362676 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.362565 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5pbn\" (UniqueName: \"kubernetes.io/projected/af9e5f62-ece3-4bf9-98df-e2fd243224e1-kube-api-access-r5pbn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:55:58.362676 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.362616 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:55:58.363001 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.362975 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:55:58.363042 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.362975 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:55:58.372747 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.372724 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5pbn\" (UniqueName: \"kubernetes.io/projected/af9e5f62-ece3-4bf9-98df-e2fd243224e1-kube-api-access-r5pbn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:55:58.511697 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.511667 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:55:58.635182 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.635156 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc"] Jul 15 18:55:58.637168 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:55:58.637140 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9e5f62_ece3_4bf9_98df_e2fd243224e1.slice/crio-68d3d707fda54b777bef6623fa35fc007b39489176a9c9d9d6f49c5b72b31953 WatchSource:0}: Error finding container 68d3d707fda54b777bef6623fa35fc007b39489176a9c9d9d6f49c5b72b31953: Status 404 returned error can't find the container with id 68d3d707fda54b777bef6623fa35fc007b39489176a9c9d9d6f49c5b72b31953 Jul 15 18:55:58.867373 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:58.867293 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:55:59.416072 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:59.416041 2574 generic.go:356] "Generic (PLEG): container finished" podID="af9e5f62-ece3-4bf9-98df-e2fd243224e1" containerID="0c94d1c2f7adfb2fff36945f62d1dcc587e0a2c05303bfaa272abc2f1065df99" exitCode=0 Jul 15 18:55:59.416418 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:59.416132 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" event={"ID":"af9e5f62-ece3-4bf9-98df-e2fd243224e1","Type":"ContainerDied","Data":"0c94d1c2f7adfb2fff36945f62d1dcc587e0a2c05303bfaa272abc2f1065df99"} Jul 15 18:55:59.416418 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:59.416193 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" event={"ID":"af9e5f62-ece3-4bf9-98df-e2fd243224e1","Type":"ContainerStarted","Data":"68d3d707fda54b777bef6623fa35fc007b39489176a9c9d9d6f49c5b72b31953"} Jul 15 18:55:59.416595 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:55:59.416581 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 18:56:00.422473 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:00.422443 2574 generic.go:356] "Generic (PLEG): container finished" podID="af9e5f62-ece3-4bf9-98df-e2fd243224e1" containerID="e2fb70938b3200de879cc355c1da52b7159cc838edf775c558281ee6c00423ac" exitCode=0 Jul 15 18:56:00.422851 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:00.422522 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" event={"ID":"af9e5f62-ece3-4bf9-98df-e2fd243224e1","Type":"ContainerDied","Data":"e2fb70938b3200de879cc355c1da52b7159cc838edf775c558281ee6c00423ac"} Jul 15 18:56:01.428826 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:01.428766 2574 generic.go:356] "Generic (PLEG): container finished" podID="af9e5f62-ece3-4bf9-98df-e2fd243224e1" containerID="d32f66db345c6ceb78f9d8745357e85b6402b809aca3df090a602cf2d55c63e5" exitCode=0 Jul 15 18:56:01.429223 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:01.428852 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" event={"ID":"af9e5f62-ece3-4bf9-98df-e2fd243224e1","Type":"ContainerDied","Data":"d32f66db345c6ceb78f9d8745357e85b6402b809aca3df090a602cf2d55c63e5"} Jul 15 18:56:02.554291 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:02.554266 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:56:02.594340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:02.594312 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5pbn\" (UniqueName: \"kubernetes.io/projected/af9e5f62-ece3-4bf9-98df-e2fd243224e1-kube-api-access-r5pbn\") pod \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " Jul 15 18:56:02.594492 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:02.594417 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-bundle\") pod \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " Jul 15 18:56:02.594492 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:02.594437 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-util\") pod \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\" (UID: \"af9e5f62-ece3-4bf9-98df-e2fd243224e1\") " Jul 15 18:56:02.595092 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:02.595067 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-bundle" (OuterVolumeSpecName: "bundle") pod "af9e5f62-ece3-4bf9-98df-e2fd243224e1" (UID: "af9e5f62-ece3-4bf9-98df-e2fd243224e1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:56:02.596437 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:02.596406 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af9e5f62-ece3-4bf9-98df-e2fd243224e1-kube-api-access-r5pbn" (OuterVolumeSpecName: "kube-api-access-r5pbn") pod "af9e5f62-ece3-4bf9-98df-e2fd243224e1" (UID: "af9e5f62-ece3-4bf9-98df-e2fd243224e1"). InnerVolumeSpecName "kube-api-access-r5pbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:56:02.600420 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:02.600364 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-util" (OuterVolumeSpecName: "util") pod "af9e5f62-ece3-4bf9-98df-e2fd243224e1" (UID: "af9e5f62-ece3-4bf9-98df-e2fd243224e1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:56:02.695022 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:02.694921 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-bundle\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:56:02.695022 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:02.694965 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af9e5f62-ece3-4bf9-98df-e2fd243224e1-util\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:56:02.695022 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:02.694976 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5pbn\" (UniqueName: \"kubernetes.io/projected/af9e5f62-ece3-4bf9-98df-e2fd243224e1-kube-api-access-r5pbn\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:56:03.439036 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:03.439001 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" event={"ID":"af9e5f62-ece3-4bf9-98df-e2fd243224e1","Type":"ContainerDied","Data":"68d3d707fda54b777bef6623fa35fc007b39489176a9c9d9d6f49c5b72b31953"} Jul 15 18:56:03.439200 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:03.439051 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68d3d707fda54b777bef6623fa35fc007b39489176a9c9d9d6f49c5b72b31953" Jul 15 18:56:03.439200 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:03.439017 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc22sc" Jul 15 18:56:14.741104 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:14.741067 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:14.847704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:14.847655 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:15.352467 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.352428 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:15.682292 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.682200 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:15.787556 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.787514 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:15.824453 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.824417 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw"] Jul 15 18:56:15.824759 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.824742 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af9e5f62-ece3-4bf9-98df-e2fd243224e1" containerName="util" Jul 15 18:56:15.824860 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.824762 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af9e5f62-ece3-4bf9-98df-e2fd243224e1" containerName="util" Jul 15 18:56:15.824860 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.824799 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af9e5f62-ece3-4bf9-98df-e2fd243224e1" containerName="extract" Jul 15 18:56:15.824860 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.824809 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af9e5f62-ece3-4bf9-98df-e2fd243224e1" containerName="extract" Jul 15 18:56:15.824860 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.824836 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af9e5f62-ece3-4bf9-98df-e2fd243224e1" containerName="pull" Jul 15 18:56:15.824860 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.824845 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af9e5f62-ece3-4bf9-98df-e2fd243224e1" containerName="pull" Jul 15 18:56:15.825099 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.824923 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="af9e5f62-ece3-4bf9-98df-e2fd243224e1" containerName="extract" Jul 15 18:56:15.827800 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.827769 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:15.833194 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.833168 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 18:56:15.833311 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.833285 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ssvhg\"" Jul 15 18:56:15.833373 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.833355 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 18:56:15.833640 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.833621 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 18:56:15.834504 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.834463 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 18:56:15.852837 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.852814 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw"] Jul 15 18:56:15.902427 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.902385 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpdxm\" (UniqueName: \"kubernetes.io/projected/12951936-1742-485e-a2a2-7d3b57a86f88-kube-api-access-bpdxm\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:15.902427 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.902433 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12951936-1742-485e-a2a2-7d3b57a86f88-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:15.902660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.902542 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12951936-1742-485e-a2a2-7d3b57a86f88-manifests\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:15.902660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.902568 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12951936-1742-485e-a2a2-7d3b57a86f88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:15.902660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:15.902598 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12951936-1742-485e-a2a2-7d3b57a86f88-tmp\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.003159 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.003132 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12951936-1742-485e-a2a2-7d3b57a86f88-tmp\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.003314 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.003183 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpdxm\" (UniqueName: \"kubernetes.io/projected/12951936-1742-485e-a2a2-7d3b57a86f88-kube-api-access-bpdxm\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.003314 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.003300 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12951936-1742-485e-a2a2-7d3b57a86f88-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.003404 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.003389 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12951936-1742-485e-a2a2-7d3b57a86f88-manifests\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.003453 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.003405 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12951936-1742-485e-a2a2-7d3b57a86f88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.003699 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.003677 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12951936-1742-485e-a2a2-7d3b57a86f88-manifests\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.005448 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.005415 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12951936-1742-485e-a2a2-7d3b57a86f88-tmp\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.005572 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.005552 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12951936-1742-485e-a2a2-7d3b57a86f88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.005709 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.005691 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12951936-1742-485e-a2a2-7d3b57a86f88-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.025838 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.025812 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:16.026249 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.026232 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpdxm\" (UniqueName: \"kubernetes.io/projected/12951936-1742-485e-a2a2-7d3b57a86f88-kube-api-access-bpdxm\") pod \"opendatahub-operator-controller-manager-6dbd7b8848-wd9vw\" (UID: \"12951936-1742-485e-a2a2-7d3b57a86f88\") " pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.137660 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.137626 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:16.169809 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.169770 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:16.268028 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.267950 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:16.277106 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.277074 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw"] Jul 15 18:56:16.280263 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:56:16.280237 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12951936_1742_485e_a2a2_7d3b57a86f88.slice/crio-c09e3dfd1dbe895a36fa64e61992c34bc09411effbefeec62edf6c0a7a7d6725 WatchSource:0}: Error finding container c09e3dfd1dbe895a36fa64e61992c34bc09411effbefeec62edf6c0a7a7d6725: Status 404 returned error can't find the container with id c09e3dfd1dbe895a36fa64e61992c34bc09411effbefeec62edf6c0a7a7d6725 Jul 15 18:56:16.372311 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.372273 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:16.469064 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.469031 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:16.491567 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:16.491518 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" event={"ID":"12951936-1742-485e-a2a2-7d3b57a86f88","Type":"ContainerStarted","Data":"c09e3dfd1dbe895a36fa64e61992c34bc09411effbefeec62edf6c0a7a7d6725"} Jul 15 18:56:17.190358 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:17.190317 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:17.401179 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:17.401136 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:19.507613 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:19.507578 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" event={"ID":"12951936-1742-485e-a2a2-7d3b57a86f88","Type":"ContainerStarted","Data":"6d45e6a2e6c2a4073b45db02cecdf200866af52a1f8e0c93d3d99aa0ed4ed078"} Jul 15 18:56:20.514341 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:20.514249 2574 generic.go:356] "Generic (PLEG): container finished" podID="12951936-1742-485e-a2a2-7d3b57a86f88" containerID="6d45e6a2e6c2a4073b45db02cecdf200866af52a1f8e0c93d3d99aa0ed4ed078" exitCode=0 Jul 15 18:56:20.514703 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:20.514342 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" event={"ID":"12951936-1742-485e-a2a2-7d3b57a86f88","Type":"ContainerDied","Data":"6d45e6a2e6c2a4073b45db02cecdf200866af52a1f8e0c93d3d99aa0ed4ed078"} Jul 15 18:56:21.483796 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:21.483664 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:21.520110 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:21.520077 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" event={"ID":"12951936-1742-485e-a2a2-7d3b57a86f88","Type":"ContainerStarted","Data":"9466a014da30c2ffcd96cfcf1fd75872b693aaa07dabccf4a34a66a11591fd11"} Jul 15 18:56:21.520461 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:21.520188 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:21.583618 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:21.583552 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" podStartSLOduration=2.169984066 podStartE2EDuration="6.583532786s" totalImagesPullingTime="3.327245059s" totalInitContainerRuntime="1.086303661s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:56:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:56:21.582711805 +0000 UTC m=+331.930424124" watchObservedRunningTime="2026-07-15 18:56:21.583532786 +0000 UTC m=+331.931245086" Jul 15 18:56:21.657901 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:21.657864 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:27.425313 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:27.425276 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:27.568206 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:27.568169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:32.525855 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:32.525817 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dbd7b8848-wd9vw" Jul 15 18:56:35.043513 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:35.043464 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:35.391818 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:35.391713 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:35.522447 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:35.522411 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:36.132042 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.132007 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb"] Jul 15 18:56:36.139380 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.139349 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:36.144391 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.144373 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" Jul 15 18:56:36.147876 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.147857 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 18:56:36.147981 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.147883 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 18:56:36.147981 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.147890 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 18:56:36.148139 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.148125 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 18:56:36.182384 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.182358 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb"] Jul 15 18:56:36.277069 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.277033 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/596dd6fd-f306-4cdf-ba48-3f73028221c0-tmp\") pod \"kube-auth-proxy-7cbd849f4d-98bjb\" (UID: \"596dd6fd-f306-4cdf-ba48-3f73028221c0\") " pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" Jul 15 18:56:36.277253 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.277086 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/596dd6fd-f306-4cdf-ba48-3f73028221c0-tls-certs\") pod \"kube-auth-proxy-7cbd849f4d-98bjb\" (UID: \"596dd6fd-f306-4cdf-ba48-3f73028221c0\") " pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" Jul 15 18:56:36.277253 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.277179 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czl4q\" (UniqueName: \"kubernetes.io/projected/596dd6fd-f306-4cdf-ba48-3f73028221c0-kube-api-access-czl4q\") pod \"kube-auth-proxy-7cbd849f4d-98bjb\" (UID: \"596dd6fd-f306-4cdf-ba48-3f73028221c0\") " pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" Jul 15 18:56:36.297639 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.297609 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d7995"] Jul 15 18:56:36.306159 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.306126 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-d7995" Jul 15 18:56:36.308954 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.308921 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 15 18:56:36.309077 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.308989 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-jf9t5\"" Jul 15 18:56:36.320496 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.320471 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d7995"] Jul 15 18:56:36.345910 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.345879 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:36.377698 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.377660 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/596dd6fd-f306-4cdf-ba48-3f73028221c0-tmp\") pod \"kube-auth-proxy-7cbd849f4d-98bjb\" (UID: \"596dd6fd-f306-4cdf-ba48-3f73028221c0\") " pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" Jul 15 18:56:36.377870 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.377825 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/596dd6fd-f306-4cdf-ba48-3f73028221c0-tls-certs\") pod \"kube-auth-proxy-7cbd849f4d-98bjb\" (UID: \"596dd6fd-f306-4cdf-ba48-3f73028221c0\") " pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" Jul 15 18:56:36.377941 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.377888 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czl4q\" (UniqueName: \"kubernetes.io/projected/596dd6fd-f306-4cdf-ba48-3f73028221c0-kube-api-access-czl4q\") pod \"kube-auth-proxy-7cbd849f4d-98bjb\" (UID: \"596dd6fd-f306-4cdf-ba48-3f73028221c0\") " pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" Jul 15 18:56:36.380154 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.380134 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/596dd6fd-f306-4cdf-ba48-3f73028221c0-tmp\") pod \"kube-auth-proxy-7cbd849f4d-98bjb\" (UID: \"596dd6fd-f306-4cdf-ba48-3f73028221c0\") " pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" Jul 15 18:56:36.380583 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.380563 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/596dd6fd-f306-4cdf-ba48-3f73028221c0-tls-certs\") pod \"kube-auth-proxy-7cbd849f4d-98bjb\" (UID: \"596dd6fd-f306-4cdf-ba48-3f73028221c0\") " pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" Jul 15 18:56:36.388596 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.388540 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czl4q\" (UniqueName: \"kubernetes.io/projected/596dd6fd-f306-4cdf-ba48-3f73028221c0-kube-api-access-czl4q\") pod \"kube-auth-proxy-7cbd849f4d-98bjb\" (UID: \"596dd6fd-f306-4cdf-ba48-3f73028221c0\") " pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" Jul 15 18:56:36.445455 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.445419 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:36.453318 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.453292 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" Jul 15 18:56:36.479233 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.479184 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cgmt\" (UniqueName: \"kubernetes.io/projected/d5cf65fc-bc88-4162-8f42-19760b3b9be3-kube-api-access-4cgmt\") pod \"odh-model-controller-7998c67dc4-d7995\" (UID: \"d5cf65fc-bc88-4162-8f42-19760b3b9be3\") " pod="opendatahub/odh-model-controller-7998c67dc4-d7995" Jul 15 18:56:36.479383 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.479298 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5cf65fc-bc88-4162-8f42-19760b3b9be3-cert\") pod \"odh-model-controller-7998c67dc4-d7995\" (UID: \"d5cf65fc-bc88-4162-8f42-19760b3b9be3\") " pod="opendatahub/odh-model-controller-7998c67dc4-d7995" Jul 15 18:56:36.579969 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.579940 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cgmt\" (UniqueName: \"kubernetes.io/projected/d5cf65fc-bc88-4162-8f42-19760b3b9be3-kube-api-access-4cgmt\") pod \"odh-model-controller-7998c67dc4-d7995\" (UID: \"d5cf65fc-bc88-4162-8f42-19760b3b9be3\") " pod="opendatahub/odh-model-controller-7998c67dc4-d7995" Jul 15 18:56:36.580116 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.579985 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5cf65fc-bc88-4162-8f42-19760b3b9be3-cert\") pod \"odh-model-controller-7998c67dc4-d7995\" (UID: \"d5cf65fc-bc88-4162-8f42-19760b3b9be3\") " pod="opendatahub/odh-model-controller-7998c67dc4-d7995" Jul 15 18:56:36.582553 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.582519 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5cf65fc-bc88-4162-8f42-19760b3b9be3-cert\") pod \"odh-model-controller-7998c67dc4-d7995\" (UID: \"d5cf65fc-bc88-4162-8f42-19760b3b9be3\") " pod="opendatahub/odh-model-controller-7998c67dc4-d7995" Jul 15 18:56:36.622764 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.622728 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb"] Jul 15 18:56:36.622952 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.622927 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cgmt\" (UniqueName: \"kubernetes.io/projected/d5cf65fc-bc88-4162-8f42-19760b3b9be3-kube-api-access-4cgmt\") pod \"odh-model-controller-7998c67dc4-d7995\" (UID: \"d5cf65fc-bc88-4162-8f42-19760b3b9be3\") " pod="opendatahub/odh-model-controller-7998c67dc4-d7995" Jul 15 18:56:36.625703 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:56:36.625685 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596dd6fd_f306_4cdf_ba48_3f73028221c0.slice/crio-dc31050f9798b286b2f3d52bb4d6e50b8f60ca8a084a2012d6af23561307bbac WatchSource:0}: Error finding container dc31050f9798b286b2f3d52bb4d6e50b8f60ca8a084a2012d6af23561307bbac: Status 404 returned error can't find the container with id dc31050f9798b286b2f3d52bb4d6e50b8f60ca8a084a2012d6af23561307bbac Jul 15 18:56:36.634694 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.634671 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:36.773198 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.773161 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:36.918004 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:36.917970 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-d7995" Jul 15 18:56:37.048224 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.048200 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d7995"] Jul 15 18:56:37.050266 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:56:37.050241 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5cf65fc_bc88_4162_8f42_19760b3b9be3.slice/crio-e816d026fca72e07eeea0b59847864a5430dc7b0f682984658bbcba9d80ebb33 WatchSource:0}: Error finding container e816d026fca72e07eeea0b59847864a5430dc7b0f682984658bbcba9d80ebb33: Status 404 returned error can't find the container with id e816d026fca72e07eeea0b59847864a5430dc7b0f682984658bbcba9d80ebb33 Jul 15 18:56:37.300186 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.300097 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cfwq4"] Jul 15 18:56:37.305255 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.305237 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" Jul 15 18:56:37.311696 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.311596 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 15 18:56:37.345163 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.345136 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-xpstx\"" Jul 15 18:56:37.349756 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.349732 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cfwq4"] Jul 15 18:56:37.374370 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.374291 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:37.488541 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.488504 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm8sz\" (UniqueName: \"kubernetes.io/projected/587d6702-7ab8-425d-afc1-3e60ab776316-kube-api-access-vm8sz\") pod \"model-serving-api-54d89bd5fc-cfwq4\" (UID: \"587d6702-7ab8-425d-afc1-3e60ab776316\") " pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" Jul 15 18:56:37.488696 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.488637 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/587d6702-7ab8-425d-afc1-3e60ab776316-tls-certs\") pod \"model-serving-api-54d89bd5fc-cfwq4\" (UID: \"587d6702-7ab8-425d-afc1-3e60ab776316\") " pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" Jul 15 18:56:37.589855 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.589366 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/587d6702-7ab8-425d-afc1-3e60ab776316-tls-certs\") pod \"model-serving-api-54d89bd5fc-cfwq4\" (UID: \"587d6702-7ab8-425d-afc1-3e60ab776316\") " pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" Jul 15 18:56:37.589855 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.589445 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm8sz\" (UniqueName: \"kubernetes.io/projected/587d6702-7ab8-425d-afc1-3e60ab776316-kube-api-access-vm8sz\") pod \"model-serving-api-54d89bd5fc-cfwq4\" (UID: \"587d6702-7ab8-425d-afc1-3e60ab776316\") " pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" Jul 15 18:56:37.592622 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.592595 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/587d6702-7ab8-425d-afc1-3e60ab776316-tls-certs\") pod \"model-serving-api-54d89bd5fc-cfwq4\" (UID: \"587d6702-7ab8-425d-afc1-3e60ab776316\") " pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" Jul 15 18:56:37.597880 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.597846 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" event={"ID":"596dd6fd-f306-4cdf-ba48-3f73028221c0","Type":"ContainerStarted","Data":"dc31050f9798b286b2f3d52bb4d6e50b8f60ca8a084a2012d6af23561307bbac"} Jul 15 18:56:37.599377 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.599347 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-d7995" event={"ID":"d5cf65fc-bc88-4162-8f42-19760b3b9be3","Type":"ContainerStarted","Data":"e816d026fca72e07eeea0b59847864a5430dc7b0f682984658bbcba9d80ebb33"} Jul 15 18:56:37.614926 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.614903 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm8sz\" (UniqueName: \"kubernetes.io/projected/587d6702-7ab8-425d-afc1-3e60ab776316-kube-api-access-vm8sz\") pod \"model-serving-api-54d89bd5fc-cfwq4\" (UID: \"587d6702-7ab8-425d-afc1-3e60ab776316\") " pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" Jul 15 18:56:37.615070 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.615052 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" Jul 15 18:56:37.629329 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.629298 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:37.789203 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:37.789172 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cfwq4"] Jul 15 18:56:37.791760 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:56:37.791619 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod587d6702_7ab8_425d_afc1_3e60ab776316.slice/crio-dc8c4e5fd04f78b089786af43b11ee0e84a62becca194100081bb42ef9102d61 WatchSource:0}: Error finding container dc8c4e5fd04f78b089786af43b11ee0e84a62becca194100081bb42ef9102d61: Status 404 returned error can't find the container with id dc8c4e5fd04f78b089786af43b11ee0e84a62becca194100081bb42ef9102d61 Jul 15 18:56:38.356797 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.356732 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:38.394972 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.394288 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-prmbm"] Jul 15 18:56:38.398339 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.398312 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.401633 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.401606 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 18:56:38.401767 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.401637 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 18:56:38.401767 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.401645 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vbjzr\"" Jul 15 18:56:38.402207 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.402128 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 18:56:38.403223 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.403201 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 18:56:38.419974 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.419946 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-prmbm"] Jul 15 18:56:38.486355 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.486227 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:38.498292 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.498141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-istio-token\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.498292 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.498202 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.498292 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.498247 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-cacerts\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.498648 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.498332 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.498648 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.498375 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.498648 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.498418 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vvls\" (UniqueName: \"kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-kube-api-access-2vvls\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.498648 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.498463 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52e0aec-f875-401f-be91-767f5fb95be1-local-certs\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.599700 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.599653 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.599904 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.599726 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vvls\" (UniqueName: \"kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-kube-api-access-2vvls\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.599904 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.599776 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52e0aec-f875-401f-be91-767f5fb95be1-local-certs\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.599904 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.599848 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-istio-token\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.599904 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.599884 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.600121 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.599923 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-cacerts\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.600121 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.599961 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.600372 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.600349 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.603752 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.603725 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.606023 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.605971 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52e0aec-f875-401f-be91-767f5fb95be1-local-certs\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.606134 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.606050 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.606381 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.606359 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-cacerts\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.611465 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.611347 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" event={"ID":"587d6702-7ab8-425d-afc1-3e60ab776316","Type":"ContainerStarted","Data":"dc8c4e5fd04f78b089786af43b11ee0e84a62becca194100081bb42ef9102d61"} Jul 15 18:56:38.614354 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.614327 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vvls\" (UniqueName: \"kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-kube-api-access-2vvls\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.616184 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.616154 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-istio-token\") pod \"istiod-openshift-gateway-7574459656-prmbm\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.712429 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.712394 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:38.906587 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:38.906554 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-prmbm"] Jul 15 18:56:38.910386 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:56:38.910294 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52e0aec_f875_401f_be91_767f5fb95be1.slice/crio-4a4ca7ffda9d10c3b88604a99d35a20928596433e52a39c3c937c59c8705d99a WatchSource:0}: Error finding container 4a4ca7ffda9d10c3b88604a99d35a20928596433e52a39c3c937c59c8705d99a: Status 404 returned error can't find the container with id 4a4ca7ffda9d10c3b88604a99d35a20928596433e52a39c3c937c59c8705d99a Jul 15 18:56:39.618007 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:39.617972 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" event={"ID":"f52e0aec-f875-401f-be91-767f5fb95be1","Type":"ContainerStarted","Data":"4a4ca7ffda9d10c3b88604a99d35a20928596433e52a39c3c937c59c8705d99a"} Jul 15 18:56:39.912738 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:39.912583 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:42.701799 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:42.701743 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 18:56:42.702166 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:42.701848 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 18:56:43.592251 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.592215 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:43.638996 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.638957 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" event={"ID":"587d6702-7ab8-425d-afc1-3e60ab776316","Type":"ContainerStarted","Data":"16be2bcb878339fad89b9bcdf7917ac7962795e4818b6ba89842f1be3f6d5754"} Jul 15 18:56:43.639196 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.639063 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" Jul 15 18:56:43.640749 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.640722 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" event={"ID":"596dd6fd-f306-4cdf-ba48-3f73028221c0","Type":"ContainerStarted","Data":"3153afde4f1b7d2b1c072bea0c604728d51de43b817b433c70d9edd5aa655e54"} Jul 15 18:56:43.642509 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.642486 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" event={"ID":"f52e0aec-f875-401f-be91-767f5fb95be1","Type":"ContainerStarted","Data":"f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d"} Jul 15 18:56:43.642598 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.642529 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:43.644072 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.644048 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-d7995" event={"ID":"d5cf65fc-bc88-4162-8f42-19760b3b9be3","Type":"ContainerStarted","Data":"9cfc05f73a201790ccd5eea8bc6a8681409e176d9dc6f74d04f9c6499137d1ad"} Jul 15 18:56:43.644194 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.644182 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-d7995" Jul 15 18:56:43.644433 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.644419 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:43.660266 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.660216 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" podStartSLOduration=1.820460572 podStartE2EDuration="6.660200211s" totalImagesPullingTime="4.839739639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:56:43.659033199 +0000 UTC m=+354.006745520" watchObservedRunningTime="2026-07-15 18:56:43.660200211 +0000 UTC m=+354.007912509" Jul 15 18:56:43.679878 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.679688 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7cbd849f4d-98bjb" podStartSLOduration=1.670967761 podStartE2EDuration="7.679666866s" totalImagesPullingTime="6.008699105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:56:43.679379524 +0000 UTC m=+354.027091822" watchObservedRunningTime="2026-07-15 18:56:43.679666866 +0000 UTC m=+354.027379143" Jul 15 18:56:43.701116 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.700899 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:43.711050 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.710998 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" podStartSLOduration=1.9231550720000001 podStartE2EDuration="5.71098214s" totalImagesPullingTime="3.787827068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:56:43.710647245 +0000 UTC m=+354.058359544" watchObservedRunningTime="2026-07-15 18:56:43.71098214 +0000 UTC m=+354.058694438" Jul 15 18:56:43.736813 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.736739 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-d7995" podStartSLOduration=2.152420167 podStartE2EDuration="7.736718057s" totalImagesPullingTime="5.58429789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:56:43.735827603 +0000 UTC m=+354.083539902" watchObservedRunningTime="2026-07-15 18:56:43.736718057 +0000 UTC m=+354.084430359" Jul 15 18:56:43.776526 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:43.776492 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:46.862408 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:46.862375 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz"] Jul 15 18:56:46.865938 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:46.865921 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:46.873305 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:46.873277 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"lws-controller-manager-7db65df97c-mj9pz\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" podUID="881ce289-489e-40d4-bfe1-4d9720c4601f" pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:46.873305 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:46.873294 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"lws-controller-manager-dockercfg-vqxhs\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vqxhs\"" type="*v1.Secret" Jul 15 18:56:46.873878 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:46.873855 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"lws-manager-config\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" type="*v1.ConfigMap" Jul 15 18:56:46.873953 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:46.873882 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 15 18:56:46.873953 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:46.873887 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-cert\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" type="*v1.Secret" Jul 15 18:56:46.874066 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:46.874046 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 15 18:56:46.874112 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:46.874067 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"webhook-server-cert\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" type="*v1.Secret" Jul 15 18:56:46.884322 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:46.884296 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz"] Jul 15 18:56:46.969052 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:46.969014 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/881ce289-489e-40d4-bfe1-4d9720c4601f-manager-config\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:46.969266 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:46.969065 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/881ce289-489e-40d4-bfe1-4d9720c4601f-metrics-cert\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:46.969266 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:46.969134 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkck9\" (UniqueName: \"kubernetes.io/projected/881ce289-489e-40d4-bfe1-4d9720c4601f-kube-api-access-dkck9\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:46.969266 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:46.969217 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/881ce289-489e-40d4-bfe1-4d9720c4601f-cert\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:47.070179 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:47.070136 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkck9\" (UniqueName: \"kubernetes.io/projected/881ce289-489e-40d4-bfe1-4d9720c4601f-kube-api-access-dkck9\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:47.070367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:47.070197 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/881ce289-489e-40d4-bfe1-4d9720c4601f-cert\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:47.070367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:47.070224 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/881ce289-489e-40d4-bfe1-4d9720c4601f-manager-config\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:47.070367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:47.070262 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/881ce289-489e-40d4-bfe1-4d9720c4601f-metrics-cert\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:47.878147 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:47.878106 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:47.885639 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:47.885604 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 18:56:47.891777 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:47.891755 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/881ce289-489e-40d4-bfe1-4d9720c4601f-manager-config\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:47.907729 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:47.907698 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 18:56:47.913087 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:47.913066 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/881ce289-489e-40d4-bfe1-4d9720c4601f-cert\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:48.001147 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.001113 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:48.043736 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.043702 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 18:56:48.053220 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.053186 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/881ce289-489e-40d4-bfe1-4d9720c4601f-metrics-cert\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:48.084731 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:48.084680 2574 projected.go:291] Couldn't get configMap openshift-lws-operator/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 15 18:56:48.245286 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.245194 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:48.271764 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.271726 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vqxhs\"" Jul 15 18:56:48.347522 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.347491 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:48.349156 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.349139 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 18:56:48.460588 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.460547 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:48.462697 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.462675 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:56:48.465676 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:48.465661 2574 projected.go:196] Error preparing data for projected volume kube-api-access-dkck9 for pod openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz: failed to sync configmap cache: timed out waiting for the condition Jul 15 18:56:48.465798 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:48.465719 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/881ce289-489e-40d4-bfe1-4d9720c4601f-kube-api-access-dkck9 podName:881ce289-489e-40d4-bfe1-4d9720c4601f nodeName:}" failed. No retries permitted until 2026-07-15 18:56:48.965703714 +0000 UTC m=+359.313416004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dkck9" (UniqueName: "kubernetes.io/projected/881ce289-489e-40d4-bfe1-4d9720c4601f-kube-api-access-dkck9") pod "lws-controller-manager-7db65df97c-mj9pz" (UID: "881ce289-489e-40d4-bfe1-4d9720c4601f") : failed to sync configmap cache: timed out waiting for the condition Jul 15 18:56:48.565520 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.565482 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:56:48.987286 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.987175 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkck9\" (UniqueName: \"kubernetes.io/projected/881ce289-489e-40d4-bfe1-4d9720c4601f-kube-api-access-dkck9\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:48.989873 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:48.989847 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkck9\" (UniqueName: \"kubernetes.io/projected/881ce289-489e-40d4-bfe1-4d9720c4601f-kube-api-access-dkck9\") pod \"lws-controller-manager-7db65df97c-mj9pz\" (UID: \"881ce289-489e-40d4-bfe1-4d9720c4601f\") " pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:49.275593 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:49.275564 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:49.408523 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:49.408491 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz"] Jul 15 18:56:49.410409 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:56:49.410383 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881ce289_489e_40d4_bfe1_4d9720c4601f.slice/crio-a843121cad1a86872e399c81fb34ec9d2c452daea612f3f164ccddd4e7b703fc WatchSource:0}: Error finding container a843121cad1a86872e399c81fb34ec9d2c452daea612f3f164ccddd4e7b703fc: Status 404 returned error can't find the container with id a843121cad1a86872e399c81fb34ec9d2c452daea612f3f164ccddd4e7b703fc Jul 15 18:56:49.670217 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:49.670132 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" event={"ID":"881ce289-489e-40d4-bfe1-4d9720c4601f","Type":"ContainerStarted","Data":"a843121cad1a86872e399c81fb34ec9d2c452daea612f3f164ccddd4e7b703fc"} Jul 15 18:56:50.289979 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.289938 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-prmbm"] Jul 15 18:56:50.290510 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.290177 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" podUID="f52e0aec-f875-401f-be91-767f5fb95be1" containerName="discovery" containerID="cri-o://f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d" gracePeriod=30 Jul 15 18:56:50.656836 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.656810 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:50.675199 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.675169 2574 generic.go:356] "Generic (PLEG): container finished" podID="f52e0aec-f875-401f-be91-767f5fb95be1" containerID="f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d" exitCode=0 Jul 15 18:56:50.675329 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.675206 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" Jul 15 18:56:50.675329 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.675259 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" event={"ID":"f52e0aec-f875-401f-be91-767f5fb95be1","Type":"ContainerDied","Data":"f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d"} Jul 15 18:56:50.675437 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.675331 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-prmbm" event={"ID":"f52e0aec-f875-401f-be91-767f5fb95be1","Type":"ContainerDied","Data":"4a4ca7ffda9d10c3b88604a99d35a20928596433e52a39c3c937c59c8705d99a"} Jul 15 18:56:50.675437 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.675361 2574 scope.go:117] "RemoveContainer" containerID="f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d" Jul 15 18:56:50.677009 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.676988 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" event={"ID":"881ce289-489e-40d4-bfe1-4d9720c4601f","Type":"ContainerStarted","Data":"34985ec15aee02bcf54e111551f63497212c2f4c819f7f9d0baa9f289048ad9c"} Jul 15 18:56:50.677160 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.677140 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:56:50.683951 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.683927 2574 scope.go:117] "RemoveContainer" containerID="f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d" Jul 15 18:56:50.684346 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:50.684321 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d\": container with ID starting with f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d not found: ID does not exist" containerID="f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d" Jul 15 18:56:50.684437 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.684360 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d"} err="failed to get container status \"f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d\": rpc error: code = NotFound desc = could not find container \"f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d\": container with ID starting with f323df4274525439a0bbedcffd407a6698b68da174fd7e2d407c74462d72125d not found: ID does not exist" Jul 15 18:56:50.703340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.703313 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-dns-cert\") pod \"f52e0aec-f875-401f-be91-767f5fb95be1\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " Jul 15 18:56:50.703424 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.703380 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-kubeconfig\") pod \"f52e0aec-f875-401f-be91-767f5fb95be1\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " Jul 15 18:56:50.703549 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.703531 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-cacerts\") pod \"f52e0aec-f875-401f-be91-767f5fb95be1\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " Jul 15 18:56:50.703613 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.703591 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-ca-configmap\") pod \"f52e0aec-f875-401f-be91-767f5fb95be1\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " Jul 15 18:56:50.703658 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.703642 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52e0aec-f875-401f-be91-767f5fb95be1-local-certs\") pod \"f52e0aec-f875-401f-be91-767f5fb95be1\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " Jul 15 18:56:50.703723 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.703679 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-istio-token\") pod \"f52e0aec-f875-401f-be91-767f5fb95be1\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " Jul 15 18:56:50.703777 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.703732 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vvls\" (UniqueName: \"kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-kube-api-access-2vvls\") pod \"f52e0aec-f875-401f-be91-767f5fb95be1\" (UID: \"f52e0aec-f875-401f-be91-767f5fb95be1\") " Jul 15 18:56:50.704015 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.703990 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f52e0aec-f875-401f-be91-767f5fb95be1" (UID: "f52e0aec-f875-401f-be91-767f5fb95be1"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:56:50.706119 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.706084 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f52e0aec-f875-401f-be91-767f5fb95be1" (UID: "f52e0aec-f875-401f-be91-767f5fb95be1"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 18:56:50.706119 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.706115 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52e0aec-f875-401f-be91-767f5fb95be1-local-certs" (OuterVolumeSpecName: "local-certs") pod "f52e0aec-f875-401f-be91-767f5fb95be1" (UID: "f52e0aec-f875-401f-be91-767f5fb95be1"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:56:50.706294 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.706260 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-kube-api-access-2vvls" (OuterVolumeSpecName: "kube-api-access-2vvls") pod "f52e0aec-f875-401f-be91-767f5fb95be1" (UID: "f52e0aec-f875-401f-be91-767f5fb95be1"). InnerVolumeSpecName "kube-api-access-2vvls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:56:50.706294 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.706270 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-cacerts" (OuterVolumeSpecName: "cacerts") pod "f52e0aec-f875-401f-be91-767f5fb95be1" (UID: "f52e0aec-f875-401f-be91-767f5fb95be1"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 18:56:50.706450 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.706434 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f52e0aec-f875-401f-be91-767f5fb95be1" (UID: "f52e0aec-f875-401f-be91-767f5fb95be1"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 18:56:50.706512 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.706452 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-istio-token" (OuterVolumeSpecName: "istio-token") pod "f52e0aec-f875-401f-be91-767f5fb95be1" (UID: "f52e0aec-f875-401f-be91-767f5fb95be1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:56:50.715746 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.715705 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" podStartSLOduration=3.571425086 podStartE2EDuration="4.715691s" totalImagesPullingTime="1.144265914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:56:50.714888936 +0000 UTC m=+361.062601235" watchObservedRunningTime="2026-07-15 18:56:50.715691 +0000 UTC m=+361.063403278" Jul 15 18:56:50.805043 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.804957 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vvls\" (UniqueName: \"kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-kube-api-access-2vvls\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:56:50.805043 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.804989 2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-dns-cert\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:56:50.805043 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.805002 2574 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-istio-kubeconfig\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:56:50.805043 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.805028 2574 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f52e0aec-f875-401f-be91-767f5fb95be1-cacerts\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:56:50.805043 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.805045 2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f52e0aec-f875-401f-be91-767f5fb95be1-istio-csr-ca-configmap\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:56:50.805301 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.805057 2574 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52e0aec-f875-401f-be91-767f5fb95be1-local-certs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:56:50.805301 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:50.805068 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52e0aec-f875-401f-be91-767f5fb95be1-istio-token\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:56:50.995376 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:56:50.995346 2574 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52e0aec_f875_401f_be91_767f5fb95be1.slice\": RecentStats: unable to find data in memory cache]" Jul 15 18:56:51.000397 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:51.000374 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-prmbm"] Jul 15 18:56:51.010772 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:51.010737 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-prmbm"] Jul 15 18:56:52.205970 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:52.205940 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f52e0aec-f875-401f-be91-767f5fb95be1" path="/var/lib/kubelet/pods/f52e0aec-f875-401f-be91-767f5fb95be1/volumes" Jul 15 18:56:54.650676 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:54.650641 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-d7995" Jul 15 18:56:54.652861 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:56:54.652839 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-cfwq4" Jul 15 18:57:00.865627 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:00.865578 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:57:01.684927 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:01.684900 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7db65df97c-mj9pz" Jul 15 18:57:02.714795 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:02.714743 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:57:02.770941 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:02.770910 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:57:12.664672 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:12.664625 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:57:36.913323 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:36.913287 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:57:37.014239 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:37.014206 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:57:37.133922 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:37.133885 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:57:37.878683 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:37.878639 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:57:38.082088 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:38.082054 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:57:52.679507 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.679476 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4"] Jul 15 18:57:52.679994 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.679965 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f52e0aec-f875-401f-be91-767f5fb95be1" containerName="discovery" Jul 15 18:57:52.679994 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.679984 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f52e0aec-f875-401f-be91-767f5fb95be1" containerName="discovery" Jul 15 18:57:52.680108 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.680055 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f52e0aec-f875-401f-be91-767f5fb95be1" containerName="discovery" Jul 15 18:57:52.684313 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.684291 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:52.686724 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.686702 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 18:57:52.686837 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.686748 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 18:57:52.686837 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.686814 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4wzkg\"" Jul 15 18:57:52.689892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.689871 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4"] Jul 15 18:57:52.704194 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.704175 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:52.704305 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.704227 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:52.704305 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.704289 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv7sn\" (UniqueName: \"kubernetes.io/projected/06d1daf3-2997-459d-92ff-b0932eb82af8-kube-api-access-kv7sn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:52.805606 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.805577 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:52.805712 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.805615 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv7sn\" (UniqueName: \"kubernetes.io/projected/06d1daf3-2997-459d-92ff-b0932eb82af8-kube-api-access-kv7sn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:52.805712 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.805644 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:52.805962 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.805945 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:52.806002 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.805987 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:52.813342 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.813321 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv7sn\" (UniqueName: \"kubernetes.io/projected/06d1daf3-2997-459d-92ff-b0932eb82af8-kube-api-access-kv7sn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:52.994523 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:52.994449 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:53.077087 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.077057 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n"] Jul 15 18:57:53.080976 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.080949 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:53.088657 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.088632 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n"] Jul 15 18:57:53.107427 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.107398 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:53.107537 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.107454 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:53.107537 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.107514 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-299mk\" (UniqueName: \"kubernetes.io/projected/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-kube-api-access-299mk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:53.125597 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.125577 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4"] Jul 15 18:57:53.127343 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:57:53.127318 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06d1daf3_2997_459d_92ff_b0932eb82af8.slice/crio-7ff6bffa9a0361a039bbb343604bb416891f35843340dcfa69a279a455bc1ad3 WatchSource:0}: Error finding container 7ff6bffa9a0361a039bbb343604bb416891f35843340dcfa69a279a455bc1ad3: Status 404 returned error can't find the container with id 7ff6bffa9a0361a039bbb343604bb416891f35843340dcfa69a279a455bc1ad3 Jul 15 18:57:53.208539 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.208499 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:53.208667 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.208555 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:53.208667 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.208578 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-299mk\" (UniqueName: \"kubernetes.io/projected/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-kube-api-access-299mk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:53.208919 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.208887 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:53.209028 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.209004 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:53.220678 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.220641 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-299mk\" (UniqueName: \"kubernetes.io/projected/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-kube-api-access-299mk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:53.393325 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.393296 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:53.481508 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.481475 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526"] Jul 15 18:57:53.485099 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.485079 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:53.491689 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.491621 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526"] Jul 15 18:57:53.511568 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.511542 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2pw8\" (UniqueName: \"kubernetes.io/projected/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-kube-api-access-p2pw8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:53.511692 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.511591 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:53.511692 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.511621 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:53.518428 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.518406 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n"] Jul 15 18:57:53.520199 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:57:53.520176 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fc5d28f_47ad_4c15_a291_210c00fc1cd7.slice/crio-e9c77d4c059925d9d70271765defa801c17d3c20a960cf47dc1c3b485727e5e5 WatchSource:0}: Error finding container e9c77d4c059925d9d70271765defa801c17d3c20a960cf47dc1c3b485727e5e5: Status 404 returned error can't find the container with id e9c77d4c059925d9d70271765defa801c17d3c20a960cf47dc1c3b485727e5e5 Jul 15 18:57:53.612336 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.612310 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2pw8\" (UniqueName: \"kubernetes.io/projected/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-kube-api-access-p2pw8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:53.612460 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.612360 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:53.612521 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.612487 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:53.612657 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.612641 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:53.612865 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.612845 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:53.621096 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.621078 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2pw8\" (UniqueName: \"kubernetes.io/projected/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-kube-api-access-p2pw8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:53.796928 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.796903 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:53.882311 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.882283 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s"] Jul 15 18:57:53.885996 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.885970 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:53.895121 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.895089 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s"] Jul 15 18:57:53.915593 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.915570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpgxr\" (UniqueName: \"kubernetes.io/projected/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-kube-api-access-vpgxr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:53.915725 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.915615 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:53.915725 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.915636 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:53.927980 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.927959 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526"] Jul 15 18:57:53.930116 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:57:53.930093 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e0d4e3_2398_41f3_a06e_c3c3afaa5022.slice/crio-218de9c9033a78d46a6915e2a93d4287a4ee45be869424779758f33a658b5f4f WatchSource:0}: Error finding container 218de9c9033a78d46a6915e2a93d4287a4ee45be869424779758f33a658b5f4f: Status 404 returned error can't find the container with id 218de9c9033a78d46a6915e2a93d4287a4ee45be869424779758f33a658b5f4f Jul 15 18:57:53.952913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.952891 2574 generic.go:356] "Generic (PLEG): container finished" podID="9fc5d28f-47ad-4c15-a291-210c00fc1cd7" containerID="76dc4f00740be1a8020c65b027910485727dbcbdec8a574f78fe6960226bd610" exitCode=0 Jul 15 18:57:53.953026 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.952972 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" event={"ID":"9fc5d28f-47ad-4c15-a291-210c00fc1cd7","Type":"ContainerDied","Data":"76dc4f00740be1a8020c65b027910485727dbcbdec8a574f78fe6960226bd610"} Jul 15 18:57:53.953086 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.953024 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" event={"ID":"9fc5d28f-47ad-4c15-a291-210c00fc1cd7","Type":"ContainerStarted","Data":"e9c77d4c059925d9d70271765defa801c17d3c20a960cf47dc1c3b485727e5e5"} Jul 15 18:57:53.954282 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.954262 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" event={"ID":"60e0d4e3-2398-41f3-a06e-c3c3afaa5022","Type":"ContainerStarted","Data":"218de9c9033a78d46a6915e2a93d4287a4ee45be869424779758f33a658b5f4f"} Jul 15 18:57:53.955514 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.955496 2574 generic.go:356] "Generic (PLEG): container finished" podID="06d1daf3-2997-459d-92ff-b0932eb82af8" containerID="6dd74c38626f4b914eedfe97a4dbba29b1d0b2c05486a1c50c3969e25f55615a" exitCode=0 Jul 15 18:57:53.955592 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.955535 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" event={"ID":"06d1daf3-2997-459d-92ff-b0932eb82af8","Type":"ContainerDied","Data":"6dd74c38626f4b914eedfe97a4dbba29b1d0b2c05486a1c50c3969e25f55615a"} Jul 15 18:57:53.955592 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:53.955558 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" event={"ID":"06d1daf3-2997-459d-92ff-b0932eb82af8","Type":"ContainerStarted","Data":"7ff6bffa9a0361a039bbb343604bb416891f35843340dcfa69a279a455bc1ad3"} Jul 15 18:57:54.016337 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.016307 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpgxr\" (UniqueName: \"kubernetes.io/projected/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-kube-api-access-vpgxr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:54.016492 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.016374 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:54.016492 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.016403 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:54.016750 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.016732 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:54.016878 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.016855 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:54.024832 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.024810 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpgxr\" (UniqueName: \"kubernetes.io/projected/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-kube-api-access-vpgxr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:54.197163 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.197086 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:54.327944 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.327918 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s"] Jul 15 18:57:54.329860 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:57:54.329840 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f8dc6a_3ad1_4dad_a529_a2d9616e8274.slice/crio-e3601f93896333de29671a9e5c2f70c551e0a63b1659bd9783687420292b2e86 WatchSource:0}: Error finding container e3601f93896333de29671a9e5c2f70c551e0a63b1659bd9783687420292b2e86: Status 404 returned error can't find the container with id e3601f93896333de29671a9e5c2f70c551e0a63b1659bd9783687420292b2e86 Jul 15 18:57:54.961941 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.961913 2574 generic.go:356] "Generic (PLEG): container finished" podID="9fc5d28f-47ad-4c15-a291-210c00fc1cd7" containerID="56505125a78b873c6cd740d8a9651c4e8b409e2965e24ae3af1a95f28426f3be" exitCode=0 Jul 15 18:57:54.962267 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.962003 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" event={"ID":"9fc5d28f-47ad-4c15-a291-210c00fc1cd7","Type":"ContainerDied","Data":"56505125a78b873c6cd740d8a9651c4e8b409e2965e24ae3af1a95f28426f3be"} Jul 15 18:57:54.963674 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.963646 2574 generic.go:356] "Generic (PLEG): container finished" podID="60e0d4e3-2398-41f3-a06e-c3c3afaa5022" containerID="26e86462b6d3320d0f4f1d0afc16442614323dbaff4986956e361ffe2ba703fb" exitCode=0 Jul 15 18:57:54.963741 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.963705 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" event={"ID":"60e0d4e3-2398-41f3-a06e-c3c3afaa5022","Type":"ContainerDied","Data":"26e86462b6d3320d0f4f1d0afc16442614323dbaff4986956e361ffe2ba703fb"} Jul 15 18:57:54.965599 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.965574 2574 generic.go:356] "Generic (PLEG): container finished" podID="06d1daf3-2997-459d-92ff-b0932eb82af8" containerID="daacc65e04eea2ad50a17252d82b324a09475ab4be26b2ff3fb049866dc24e06" exitCode=0 Jul 15 18:57:54.965708 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.965670 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" event={"ID":"06d1daf3-2997-459d-92ff-b0932eb82af8","Type":"ContainerDied","Data":"daacc65e04eea2ad50a17252d82b324a09475ab4be26b2ff3fb049866dc24e06"} Jul 15 18:57:54.967295 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.967275 2574 generic.go:356] "Generic (PLEG): container finished" podID="77f8dc6a-3ad1-4dad-a529-a2d9616e8274" containerID="f9786163ae8e36d39ca2a0c2fb9e4425514ec37f0f6eb5badca8b6c95d79577e" exitCode=0 Jul 15 18:57:54.967403 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.967306 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" event={"ID":"77f8dc6a-3ad1-4dad-a529-a2d9616e8274","Type":"ContainerDied","Data":"f9786163ae8e36d39ca2a0c2fb9e4425514ec37f0f6eb5badca8b6c95d79577e"} Jul 15 18:57:54.967403 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:54.967346 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" event={"ID":"77f8dc6a-3ad1-4dad-a529-a2d9616e8274","Type":"ContainerStarted","Data":"e3601f93896333de29671a9e5c2f70c551e0a63b1659bd9783687420292b2e86"} Jul 15 18:57:55.973607 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:55.973531 2574 generic.go:356] "Generic (PLEG): container finished" podID="60e0d4e3-2398-41f3-a06e-c3c3afaa5022" containerID="407199c5bf84c19d17b5d361769c0b4ce587513084ba3370ce364620f22e861b" exitCode=0 Jul 15 18:57:55.974054 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:55.973614 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" event={"ID":"60e0d4e3-2398-41f3-a06e-c3c3afaa5022","Type":"ContainerDied","Data":"407199c5bf84c19d17b5d361769c0b4ce587513084ba3370ce364620f22e861b"} Jul 15 18:57:55.975975 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:55.975950 2574 generic.go:356] "Generic (PLEG): container finished" podID="06d1daf3-2997-459d-92ff-b0932eb82af8" containerID="8ca41199333e7e3e5023f53a851847d37dc09e87e133a754e7a8e476ca7007ad" exitCode=0 Jul 15 18:57:55.976073 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:55.976033 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" event={"ID":"06d1daf3-2997-459d-92ff-b0932eb82af8","Type":"ContainerDied","Data":"8ca41199333e7e3e5023f53a851847d37dc09e87e133a754e7a8e476ca7007ad"} Jul 15 18:57:55.978577 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:55.978555 2574 generic.go:356] "Generic (PLEG): container finished" podID="9fc5d28f-47ad-4c15-a291-210c00fc1cd7" containerID="07e731523638e8f701a003581b15c70adc16d97afb9d5e1cf7970d682bb38e8b" exitCode=0 Jul 15 18:57:55.978643 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:55.978606 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" event={"ID":"9fc5d28f-47ad-4c15-a291-210c00fc1cd7","Type":"ContainerDied","Data":"07e731523638e8f701a003581b15c70adc16d97afb9d5e1cf7970d682bb38e8b"} Jul 15 18:57:56.784261 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:56.784222 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:57:56.884472 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:56.884436 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:57:56.984942 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:56.984913 2574 generic.go:356] "Generic (PLEG): container finished" podID="60e0d4e3-2398-41f3-a06e-c3c3afaa5022" containerID="d420b8e706fe0bd7d4ac757cbc71a5e30e69303ced231b47b778981d753ea828" exitCode=0 Jul 15 18:57:56.985317 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:56.985001 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" event={"ID":"60e0d4e3-2398-41f3-a06e-c3c3afaa5022","Type":"ContainerDied","Data":"d420b8e706fe0bd7d4ac757cbc71a5e30e69303ced231b47b778981d753ea828"} Jul 15 18:57:56.986696 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:56.986673 2574 generic.go:356] "Generic (PLEG): container finished" podID="77f8dc6a-3ad1-4dad-a529-a2d9616e8274" containerID="dc5dccc146c20a3a68c8fb278360847f87e525b0da6d88ba86fefefda4901611" exitCode=0 Jul 15 18:57:56.986811 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:56.986754 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" event={"ID":"77f8dc6a-3ad1-4dad-a529-a2d9616e8274","Type":"ContainerDied","Data":"dc5dccc146c20a3a68c8fb278360847f87e525b0da6d88ba86fefefda4901611"} Jul 15 18:57:57.208941 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.208913 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:57.211588 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.211570 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:57.242999 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.242923 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-util\") pod \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " Jul 15 18:57:57.244300 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.244220 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-299mk\" (UniqueName: \"kubernetes.io/projected/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-kube-api-access-299mk\") pod \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " Jul 15 18:57:57.244300 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.244269 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv7sn\" (UniqueName: \"kubernetes.io/projected/06d1daf3-2997-459d-92ff-b0932eb82af8-kube-api-access-kv7sn\") pod \"06d1daf3-2997-459d-92ff-b0932eb82af8\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " Jul 15 18:57:57.244527 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.244317 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-util\") pod \"06d1daf3-2997-459d-92ff-b0932eb82af8\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " Jul 15 18:57:57.244527 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.244375 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-bundle\") pod \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\" (UID: \"9fc5d28f-47ad-4c15-a291-210c00fc1cd7\") " Jul 15 18:57:57.244527 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.244403 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-bundle\") pod \"06d1daf3-2997-459d-92ff-b0932eb82af8\" (UID: \"06d1daf3-2997-459d-92ff-b0932eb82af8\") " Jul 15 18:57:57.245753 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.245695 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-bundle" (OuterVolumeSpecName: "bundle") pod "06d1daf3-2997-459d-92ff-b0932eb82af8" (UID: "06d1daf3-2997-459d-92ff-b0932eb82af8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:57.246226 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.246164 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-bundle" (OuterVolumeSpecName: "bundle") pod "9fc5d28f-47ad-4c15-a291-210c00fc1cd7" (UID: "9fc5d28f-47ad-4c15-a291-210c00fc1cd7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:57.246438 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.246397 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-bundle\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:57:57.246438 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.246420 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-bundle\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:57:57.247648 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.247510 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-kube-api-access-299mk" (OuterVolumeSpecName: "kube-api-access-299mk") pod "9fc5d28f-47ad-4c15-a291-210c00fc1cd7" (UID: "9fc5d28f-47ad-4c15-a291-210c00fc1cd7"). InnerVolumeSpecName "kube-api-access-299mk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:57:57.248360 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.248327 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06d1daf3-2997-459d-92ff-b0932eb82af8-kube-api-access-kv7sn" (OuterVolumeSpecName: "kube-api-access-kv7sn") pod "06d1daf3-2997-459d-92ff-b0932eb82af8" (UID: "06d1daf3-2997-459d-92ff-b0932eb82af8"). InnerVolumeSpecName "kube-api-access-kv7sn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:57:57.252686 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.252647 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-util" (OuterVolumeSpecName: "util") pod "9fc5d28f-47ad-4c15-a291-210c00fc1cd7" (UID: "9fc5d28f-47ad-4c15-a291-210c00fc1cd7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:57.253688 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.253668 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-util" (OuterVolumeSpecName: "util") pod "06d1daf3-2997-459d-92ff-b0932eb82af8" (UID: "06d1daf3-2997-459d-92ff-b0932eb82af8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:57.347232 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.347156 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06d1daf3-2997-459d-92ff-b0932eb82af8-util\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:57:57.347232 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.347184 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-util\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:57:57.347232 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.347193 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-299mk\" (UniqueName: \"kubernetes.io/projected/9fc5d28f-47ad-4c15-a291-210c00fc1cd7-kube-api-access-299mk\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:57:57.347232 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.347202 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv7sn\" (UniqueName: \"kubernetes.io/projected/06d1daf3-2997-459d-92ff-b0932eb82af8-kube-api-access-kv7sn\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:57:57.993453 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.993415 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" event={"ID":"06d1daf3-2997-459d-92ff-b0932eb82af8","Type":"ContainerDied","Data":"7ff6bffa9a0361a039bbb343604bb416891f35843340dcfa69a279a455bc1ad3"} Jul 15 18:57:57.993844 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.993474 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ff6bffa9a0361a039bbb343604bb416891f35843340dcfa69a279a455bc1ad3" Jul 15 18:57:57.993844 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.993428 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4" Jul 15 18:57:57.995465 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.995437 2574 generic.go:356] "Generic (PLEG): container finished" podID="77f8dc6a-3ad1-4dad-a529-a2d9616e8274" containerID="aeb47e9588cf74c96707d5b86b6c86e1cd4bdd1a75a4c2a639cdd56950f5e213" exitCode=0 Jul 15 18:57:57.995567 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.995514 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" event={"ID":"77f8dc6a-3ad1-4dad-a529-a2d9616e8274","Type":"ContainerDied","Data":"aeb47e9588cf74c96707d5b86b6c86e1cd4bdd1a75a4c2a639cdd56950f5e213"} Jul 15 18:57:57.997329 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.997309 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" Jul 15 18:57:57.997408 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.997334 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n" event={"ID":"9fc5d28f-47ad-4c15-a291-210c00fc1cd7","Type":"ContainerDied","Data":"e9c77d4c059925d9d70271765defa801c17d3c20a960cf47dc1c3b485727e5e5"} Jul 15 18:57:57.997408 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:57.997376 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9c77d4c059925d9d70271765defa801c17d3c20a960cf47dc1c3b485727e5e5" Jul 15 18:57:58.120157 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:58.120135 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:58.152729 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:58.152698 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2pw8\" (UniqueName: \"kubernetes.io/projected/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-kube-api-access-p2pw8\") pod \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " Jul 15 18:57:58.152889 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:58.152753 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-bundle\") pod \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " Jul 15 18:57:58.152889 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:58.152816 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-util\") pod \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\" (UID: \"60e0d4e3-2398-41f3-a06e-c3c3afaa5022\") " Jul 15 18:57:58.153211 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:58.153187 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-bundle" (OuterVolumeSpecName: "bundle") pod "60e0d4e3-2398-41f3-a06e-c3c3afaa5022" (UID: "60e0d4e3-2398-41f3-a06e-c3c3afaa5022"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:58.154798 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:58.154757 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-kube-api-access-p2pw8" (OuterVolumeSpecName: "kube-api-access-p2pw8") pod "60e0d4e3-2398-41f3-a06e-c3c3afaa5022" (UID: "60e0d4e3-2398-41f3-a06e-c3c3afaa5022"). InnerVolumeSpecName "kube-api-access-p2pw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:57:58.158487 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:58.158455 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-util" (OuterVolumeSpecName: "util") pod "60e0d4e3-2398-41f3-a06e-c3c3afaa5022" (UID: "60e0d4e3-2398-41f3-a06e-c3c3afaa5022"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:58.254267 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:58.254239 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-util\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:57:58.254267 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:58.254264 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2pw8\" (UniqueName: \"kubernetes.io/projected/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-kube-api-access-p2pw8\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:57:58.254267 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:58.254273 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60e0d4e3-2398-41f3-a06e-c3c3afaa5022-bundle\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:57:59.003521 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.003486 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" event={"ID":"60e0d4e3-2398-41f3-a06e-c3c3afaa5022","Type":"ContainerDied","Data":"218de9c9033a78d46a6915e2a93d4287a4ee45be869424779758f33a658b5f4f"} Jul 15 18:57:59.003946 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.003533 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526" Jul 15 18:57:59.003946 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.003546 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="218de9c9033a78d46a6915e2a93d4287a4ee45be869424779758f33a658b5f4f" Jul 15 18:57:59.138091 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.138065 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:57:59.260614 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.260532 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpgxr\" (UniqueName: \"kubernetes.io/projected/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-kube-api-access-vpgxr\") pod \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " Jul 15 18:57:59.260614 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.260581 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-bundle\") pod \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " Jul 15 18:57:59.260765 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.260642 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-util\") pod \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\" (UID: \"77f8dc6a-3ad1-4dad-a529-a2d9616e8274\") " Jul 15 18:57:59.261262 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.261232 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-bundle" (OuterVolumeSpecName: "bundle") pod "77f8dc6a-3ad1-4dad-a529-a2d9616e8274" (UID: "77f8dc6a-3ad1-4dad-a529-a2d9616e8274"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:59.262596 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.262569 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-kube-api-access-vpgxr" (OuterVolumeSpecName: "kube-api-access-vpgxr") pod "77f8dc6a-3ad1-4dad-a529-a2d9616e8274" (UID: "77f8dc6a-3ad1-4dad-a529-a2d9616e8274"). InnerVolumeSpecName "kube-api-access-vpgxr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:57:59.266269 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.266235 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-util" (OuterVolumeSpecName: "util") pod "77f8dc6a-3ad1-4dad-a529-a2d9616e8274" (UID: "77f8dc6a-3ad1-4dad-a529-a2d9616e8274"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:59.361553 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.361527 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-bundle\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:57:59.361553 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.361549 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-util\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:57:59.361553 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:57:59.361558 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpgxr\" (UniqueName: \"kubernetes.io/projected/77f8dc6a-3ad1-4dad-a529-a2d9616e8274-kube-api-access-vpgxr\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:58:00.009872 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:00.009841 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" Jul 15 18:58:00.009872 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:00.009859 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s" event={"ID":"77f8dc6a-3ad1-4dad-a529-a2d9616e8274","Type":"ContainerDied","Data":"e3601f93896333de29671a9e5c2f70c551e0a63b1659bd9783687420292b2e86"} Jul 15 18:58:00.010283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:00.009928 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3601f93896333de29671a9e5c2f70c551e0a63b1659bd9783687420292b2e86" Jul 15 18:58:06.535828 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:06.535771 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:10.334479 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:10.334432 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:12.949740 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:12.949657 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:13.716277 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:13.716240 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:14.791152 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:14.791118 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:16.788176 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:16.788127 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:17.751024 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:17.750989 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:18.328394 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:18.328355 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:19.370478 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:19.370441 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:22.352539 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:22.352504 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:22.449381 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:22.449340 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:25.743499 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:25.743457 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:25.846560 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:25.846527 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:26.641437 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641399 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k"] Jul 15 18:58:26.641812 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641779 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e0d4e3-2398-41f3-a06e-c3c3afaa5022" containerName="extract" Jul 15 18:58:26.641913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641814 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e0d4e3-2398-41f3-a06e-c3c3afaa5022" containerName="extract" Jul 15 18:58:26.641913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641834 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06d1daf3-2997-459d-92ff-b0932eb82af8" containerName="util" Jul 15 18:58:26.641913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641842 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="06d1daf3-2997-459d-92ff-b0932eb82af8" containerName="util" Jul 15 18:58:26.641913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641852 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fc5d28f-47ad-4c15-a291-210c00fc1cd7" containerName="extract" Jul 15 18:58:26.641913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641860 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fc5d28f-47ad-4c15-a291-210c00fc1cd7" containerName="extract" Jul 15 18:58:26.641913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641871 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e0d4e3-2398-41f3-a06e-c3c3afaa5022" containerName="pull" Jul 15 18:58:26.641913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641879 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e0d4e3-2398-41f3-a06e-c3c3afaa5022" containerName="pull" Jul 15 18:58:26.641913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641889 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77f8dc6a-3ad1-4dad-a529-a2d9616e8274" containerName="util" Jul 15 18:58:26.641913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641897 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f8dc6a-3ad1-4dad-a529-a2d9616e8274" containerName="util" Jul 15 18:58:26.641913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641907 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06d1daf3-2997-459d-92ff-b0932eb82af8" containerName="pull" Jul 15 18:58:26.641913 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641915 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="06d1daf3-2997-459d-92ff-b0932eb82af8" containerName="pull" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641926 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06d1daf3-2997-459d-92ff-b0932eb82af8" containerName="extract" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641933 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="06d1daf3-2997-459d-92ff-b0932eb82af8" containerName="extract" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641945 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fc5d28f-47ad-4c15-a291-210c00fc1cd7" containerName="util" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641956 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fc5d28f-47ad-4c15-a291-210c00fc1cd7" containerName="util" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641965 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77f8dc6a-3ad1-4dad-a529-a2d9616e8274" containerName="extract" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641973 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f8dc6a-3ad1-4dad-a529-a2d9616e8274" containerName="extract" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641984 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fc5d28f-47ad-4c15-a291-210c00fc1cd7" containerName="pull" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.641992 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fc5d28f-47ad-4c15-a291-210c00fc1cd7" containerName="pull" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.642002 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e0d4e3-2398-41f3-a06e-c3c3afaa5022" containerName="util" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.642011 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e0d4e3-2398-41f3-a06e-c3c3afaa5022" containerName="util" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.642021 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77f8dc6a-3ad1-4dad-a529-a2d9616e8274" containerName="pull" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.642029 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f8dc6a-3ad1-4dad-a529-a2d9616e8274" containerName="pull" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.642115 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fc5d28f-47ad-4c15-a291-210c00fc1cd7" containerName="extract" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.642128 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="77f8dc6a-3ad1-4dad-a529-a2d9616e8274" containerName="extract" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.642137 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e0d4e3-2398-41f3-a06e-c3c3afaa5022" containerName="extract" Jul 15 18:58:26.642417 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.642144 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="06d1daf3-2997-459d-92ff-b0932eb82af8" containerName="extract" Jul 15 18:58:26.647116 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.647091 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:26.647246 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.647142 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:26.649441 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.649420 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 15 18:58:26.649558 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.649423 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 15 18:58:26.649558 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.649449 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 18:58:26.649558 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.649422 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 18:58:26.650190 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.650173 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4wzkg\"" Jul 15 18:58:26.655811 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.655773 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k"] Jul 15 18:58:26.785090 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.785060 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/adbc93c7-471f-4406-beeb-e748d9cee3e8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rmv8k\" (UID: \"adbc93c7-471f-4406-beeb-e748d9cee3e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:26.785090 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.785092 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gbwf\" (UniqueName: \"kubernetes.io/projected/adbc93c7-471f-4406-beeb-e748d9cee3e8-kube-api-access-7gbwf\") pod \"kuadrant-console-plugin-66b5774985-rmv8k\" (UID: \"adbc93c7-471f-4406-beeb-e748d9cee3e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:26.785510 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.785149 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/adbc93c7-471f-4406-beeb-e748d9cee3e8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rmv8k\" (UID: \"adbc93c7-471f-4406-beeb-e748d9cee3e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:26.886474 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.886438 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/adbc93c7-471f-4406-beeb-e748d9cee3e8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rmv8k\" (UID: \"adbc93c7-471f-4406-beeb-e748d9cee3e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:26.886670 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.886500 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/adbc93c7-471f-4406-beeb-e748d9cee3e8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rmv8k\" (UID: \"adbc93c7-471f-4406-beeb-e748d9cee3e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:26.886670 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.886522 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gbwf\" (UniqueName: \"kubernetes.io/projected/adbc93c7-471f-4406-beeb-e748d9cee3e8-kube-api-access-7gbwf\") pod \"kuadrant-console-plugin-66b5774985-rmv8k\" (UID: \"adbc93c7-471f-4406-beeb-e748d9cee3e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:26.886670 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:58:26.886583 2574 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 15 18:58:26.886670 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:58:26.886660 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adbc93c7-471f-4406-beeb-e748d9cee3e8-plugin-serving-cert podName:adbc93c7-471f-4406-beeb-e748d9cee3e8 nodeName:}" failed. No retries permitted until 2026-07-15 18:58:27.386642814 +0000 UTC m=+457.734355103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/adbc93c7-471f-4406-beeb-e748d9cee3e8-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-rmv8k" (UID: "adbc93c7-471f-4406-beeb-e748d9cee3e8") : secret "plugin-serving-cert" not found Jul 15 18:58:26.887226 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.887201 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/adbc93c7-471f-4406-beeb-e748d9cee3e8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rmv8k\" (UID: \"adbc93c7-471f-4406-beeb-e748d9cee3e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:26.899131 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:26.899074 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gbwf\" (UniqueName: \"kubernetes.io/projected/adbc93c7-471f-4406-beeb-e748d9cee3e8-kube-api-access-7gbwf\") pod \"kuadrant-console-plugin-66b5774985-rmv8k\" (UID: \"adbc93c7-471f-4406-beeb-e748d9cee3e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:27.033516 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:27.033483 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:27.391519 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:27.391483 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/adbc93c7-471f-4406-beeb-e748d9cee3e8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rmv8k\" (UID: \"adbc93c7-471f-4406-beeb-e748d9cee3e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:27.393996 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:27.393972 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/adbc93c7-471f-4406-beeb-e748d9cee3e8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rmv8k\" (UID: \"adbc93c7-471f-4406-beeb-e748d9cee3e8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:27.555984 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:27.555950 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" Jul 15 18:58:27.679739 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:27.679644 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k"] Jul 15 18:58:27.682525 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:58:27.682486 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadbc93c7_471f_4406_beeb_e748d9cee3e8.slice/crio-2bade82e6f6a4b117d386a5f4548ba741440f2ebabcd864e713db8fc595778aa WatchSource:0}: Error finding container 2bade82e6f6a4b117d386a5f4548ba741440f2ebabcd864e713db8fc595778aa: Status 404 returned error can't find the container with id 2bade82e6f6a4b117d386a5f4548ba741440f2ebabcd864e713db8fc595778aa Jul 15 18:58:28.145235 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:28.145203 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" event={"ID":"adbc93c7-471f-4406-beeb-e748d9cee3e8","Type":"ContainerStarted","Data":"2bade82e6f6a4b117d386a5f4548ba741440f2ebabcd864e713db8fc595778aa"} Jul 15 18:58:29.131962 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:29.131927 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:29.190294 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:29.190257 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:37.490183 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.490105 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs"] Jul 15 18:58:37.493856 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.493832 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" Jul 15 18:58:37.496897 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.496875 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-glvv5\"" Jul 15 18:58:37.528076 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.528038 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs"] Jul 15 18:58:37.582877 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.582840 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/299e200e-59a7-4107-8e3c-805c6b8ceaa0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4xmhs\" (UID: \"299e200e-59a7-4107-8e3c-805c6b8ceaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" Jul 15 18:58:37.583023 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.582893 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lnwq\" (UniqueName: \"kubernetes.io/projected/299e200e-59a7-4107-8e3c-805c6b8ceaa0-kube-api-access-7lnwq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4xmhs\" (UID: \"299e200e-59a7-4107-8e3c-805c6b8ceaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" Jul 15 18:58:37.641376 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.641338 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:37.683318 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.683288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/299e200e-59a7-4107-8e3c-805c6b8ceaa0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4xmhs\" (UID: \"299e200e-59a7-4107-8e3c-805c6b8ceaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" Jul 15 18:58:37.683451 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.683329 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lnwq\" (UniqueName: \"kubernetes.io/projected/299e200e-59a7-4107-8e3c-805c6b8ceaa0-kube-api-access-7lnwq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4xmhs\" (UID: \"299e200e-59a7-4107-8e3c-805c6b8ceaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" Jul 15 18:58:37.683667 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.683643 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/299e200e-59a7-4107-8e3c-805c6b8ceaa0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4xmhs\" (UID: \"299e200e-59a7-4107-8e3c-805c6b8ceaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" Jul 15 18:58:37.692553 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.692524 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lnwq\" (UniqueName: \"kubernetes.io/projected/299e200e-59a7-4107-8e3c-805c6b8ceaa0-kube-api-access-7lnwq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4xmhs\" (UID: \"299e200e-59a7-4107-8e3c-805c6b8ceaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" Jul 15 18:58:37.753322 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.753295 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:37.806077 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:37.806036 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" Jul 15 18:58:39.488047 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:39.488005 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:39.592544 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:39.592500 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:40.176730 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:40.176683 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:40.279171 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:40.279131 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:41.829562 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:41.829521 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:41.929959 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:41.929921 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:43.889674 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:43.889479 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:43.951874 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:43.951838 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:46.759324 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:46.759281 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:46.859777 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:46.859738 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:50.732310 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:50.732275 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs"] Jul 15 18:58:51.959464 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:51.959439 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs"] Jul 15 18:58:51.961760 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:58:51.961732 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod299e200e_59a7_4107_8e3c_805c6b8ceaa0.slice/crio-52f1788ca9591b9433e16729208a3134f28af2dd9733b0ffd3b86203c297ce26 WatchSource:0}: Error finding container 52f1788ca9591b9433e16729208a3134f28af2dd9733b0ffd3b86203c297ce26: Status 404 returned error can't find the container with id 52f1788ca9591b9433e16729208a3134f28af2dd9733b0ffd3b86203c297ce26 Jul 15 18:58:52.277982 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:52.277947 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" event={"ID":"299e200e-59a7-4107-8e3c-805c6b8ceaa0","Type":"ContainerStarted","Data":"52f1788ca9591b9433e16729208a3134f28af2dd9733b0ffd3b86203c297ce26"} Jul 15 18:58:52.279397 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:52.279371 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" event={"ID":"adbc93c7-471f-4406-beeb-e748d9cee3e8","Type":"ContainerStarted","Data":"31cb93b271289185a5e6e1cd0d08910fc805cc28079c694640300b5be6802315"} Jul 15 18:58:52.301860 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:52.301779 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rmv8k" podStartSLOduration=2.067789431 podStartE2EDuration="26.301764766s" totalImagesPullingTime="24.233975335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:58:52.299551181 +0000 UTC m=+482.647263480" watchObservedRunningTime="2026-07-15 18:58:52.301764766 +0000 UTC m=+482.649477065" Jul 15 18:58:55.385115 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:55.385079 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:55.481422 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:55.481386 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:58:57.305683 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:57.305634 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" event={"ID":"299e200e-59a7-4107-8e3c-805c6b8ceaa0","Type":"ContainerStarted","Data":"7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1"} Jul 15 18:58:57.306176 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:57.305676 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" podUID="299e200e-59a7-4107-8e3c-805c6b8ceaa0" containerName="manager" containerID="cri-o://7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1" gracePeriod=10 Jul 15 18:58:57.306176 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:57.305764 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" Jul 15 18:58:57.326654 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:57.326606 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" podStartSLOduration=15.529612665 podStartE2EDuration="20.326593871s" totalImagesPullingTime="4.796981206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:58:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:58:57.324180571 +0000 UTC m=+487.671892889" watchObservedRunningTime="2026-07-15 18:58:57.326593871 +0000 UTC m=+487.674306170" Jul 15 18:58:57.580618 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:57.580565 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" Jul 15 18:58:57.656833 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:57.656802 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lnwq\" (UniqueName: \"kubernetes.io/projected/299e200e-59a7-4107-8e3c-805c6b8ceaa0-kube-api-access-7lnwq\") pod \"299e200e-59a7-4107-8e3c-805c6b8ceaa0\" (UID: \"299e200e-59a7-4107-8e3c-805c6b8ceaa0\") " Jul 15 18:58:57.656993 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:57.656894 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/299e200e-59a7-4107-8e3c-805c6b8ceaa0-extensions-socket-volume\") pod \"299e200e-59a7-4107-8e3c-805c6b8ceaa0\" (UID: \"299e200e-59a7-4107-8e3c-805c6b8ceaa0\") " Jul 15 18:58:57.657336 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:57.657312 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/299e200e-59a7-4107-8e3c-805c6b8ceaa0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "299e200e-59a7-4107-8e3c-805c6b8ceaa0" (UID: "299e200e-59a7-4107-8e3c-805c6b8ceaa0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:58:57.658969 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:57.658944 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/299e200e-59a7-4107-8e3c-805c6b8ceaa0-kube-api-access-7lnwq" (OuterVolumeSpecName: "kube-api-access-7lnwq") pod "299e200e-59a7-4107-8e3c-805c6b8ceaa0" (UID: "299e200e-59a7-4107-8e3c-805c6b8ceaa0"). InnerVolumeSpecName "kube-api-access-7lnwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:58:57.757490 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:57.757460 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/299e200e-59a7-4107-8e3c-805c6b8ceaa0-extensions-socket-volume\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:58:57.757490 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:57.757487 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lnwq\" (UniqueName: \"kubernetes.io/projected/299e200e-59a7-4107-8e3c-805c6b8ceaa0-kube-api-access-7lnwq\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:58:58.311069 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:58.311039 2574 generic.go:356] "Generic (PLEG): container finished" podID="299e200e-59a7-4107-8e3c-805c6b8ceaa0" containerID="7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1" exitCode=0 Jul 15 18:58:58.311425 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:58.311099 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" Jul 15 18:58:58.311425 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:58.311102 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" event={"ID":"299e200e-59a7-4107-8e3c-805c6b8ceaa0","Type":"ContainerDied","Data":"7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1"} Jul 15 18:58:58.311425 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:58.311215 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs" event={"ID":"299e200e-59a7-4107-8e3c-805c6b8ceaa0","Type":"ContainerDied","Data":"52f1788ca9591b9433e16729208a3134f28af2dd9733b0ffd3b86203c297ce26"} Jul 15 18:58:58.311425 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:58.311237 2574 scope.go:117] "RemoveContainer" containerID="7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1" Jul 15 18:58:58.319480 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:58.319461 2574 scope.go:117] "RemoveContainer" containerID="7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1" Jul 15 18:58:58.319749 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:58:58.319727 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1\": container with ID starting with 7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1 not found: ID does not exist" containerID="7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1" Jul 15 18:58:58.319845 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:58.319760 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1"} err="failed to get container status \"7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1\": rpc error: code = NotFound desc = could not find container \"7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1\": container with ID starting with 7452eda8e8e1cec26643bdaa5b2079b6bf9a51b8c8243ce06dbdcf74243179b1 not found: ID does not exist" Jul 15 18:58:58.329658 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:58.329605 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs"] Jul 15 18:58:58.332322 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:58:58.332301 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4xmhs"] Jul 15 18:59:00.206503 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:00.206472 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="299e200e-59a7-4107-8e3c-805c6b8ceaa0" path="/var/lib/kubelet/pods/299e200e-59a7-4107-8e3c-805c6b8ceaa0/volumes" Jul 15 18:59:07.263302 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:07.263274 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:07.380492 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:07.380453 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:21.896046 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:21.896001 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:22.403378 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.403342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:22.503314 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.503281 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:22.683094 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.683017 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mf5q5"] Jul 15 18:59:22.683332 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.683319 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="299e200e-59a7-4107-8e3c-805c6b8ceaa0" containerName="manager" Jul 15 18:59:22.683374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.683334 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="299e200e-59a7-4107-8e3c-805c6b8ceaa0" containerName="manager" Jul 15 18:59:22.683408 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.683395 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="299e200e-59a7-4107-8e3c-805c6b8ceaa0" containerName="manager" Jul 15 18:59:22.709210 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.709181 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mf5q5"] Jul 15 18:59:22.709351 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.709258 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mf5q5" Jul 15 18:59:22.711476 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.711448 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h2tp2\"" Jul 15 18:59:22.716455 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.716425 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:22.793561 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.793528 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:22.794611 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.794590 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-rr7gh"] Jul 15 18:59:22.797959 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.797941 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rr7gh" Jul 15 18:59:22.808116 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.808090 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rr7gh"] Jul 15 18:59:22.861865 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.861824 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8qnz\" (UniqueName: \"kubernetes.io/projected/b21b0c06-28df-48dc-9dfd-34acda9908e4-kube-api-access-s8qnz\") pod \"authorino-58cd955675-mf5q5\" (UID: \"b21b0c06-28df-48dc-9dfd-34acda9908e4\") " pod="kuadrant-system/authorino-58cd955675-mf5q5" Jul 15 18:59:22.962704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.962615 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8qnz\" (UniqueName: \"kubernetes.io/projected/b21b0c06-28df-48dc-9dfd-34acda9908e4-kube-api-access-s8qnz\") pod \"authorino-58cd955675-mf5q5\" (UID: \"b21b0c06-28df-48dc-9dfd-34acda9908e4\") " pod="kuadrant-system/authorino-58cd955675-mf5q5" Jul 15 18:59:22.962704 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.962667 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4xr9\" (UniqueName: \"kubernetes.io/projected/03bb70c7-4249-433a-8a70-b582613aa5c8-kube-api-access-q4xr9\") pod \"authorino-75d556bd77-rr7gh\" (UID: \"03bb70c7-4249-433a-8a70-b582613aa5c8\") " pod="kuadrant-system/authorino-75d556bd77-rr7gh" Jul 15 18:59:22.970433 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:22.970402 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8qnz\" (UniqueName: \"kubernetes.io/projected/b21b0c06-28df-48dc-9dfd-34acda9908e4-kube-api-access-s8qnz\") pod \"authorino-58cd955675-mf5q5\" (UID: \"b21b0c06-28df-48dc-9dfd-34acda9908e4\") " pod="kuadrant-system/authorino-58cd955675-mf5q5" Jul 15 18:59:23.018972 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:23.018947 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mf5q5" Jul 15 18:59:23.063379 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:23.063349 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4xr9\" (UniqueName: \"kubernetes.io/projected/03bb70c7-4249-433a-8a70-b582613aa5c8-kube-api-access-q4xr9\") pod \"authorino-75d556bd77-rr7gh\" (UID: \"03bb70c7-4249-433a-8a70-b582613aa5c8\") " pod="kuadrant-system/authorino-75d556bd77-rr7gh" Jul 15 18:59:23.071562 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:23.071530 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4xr9\" (UniqueName: \"kubernetes.io/projected/03bb70c7-4249-433a-8a70-b582613aa5c8-kube-api-access-q4xr9\") pod \"authorino-75d556bd77-rr7gh\" (UID: \"03bb70c7-4249-433a-8a70-b582613aa5c8\") " pod="kuadrant-system/authorino-75d556bd77-rr7gh" Jul 15 18:59:23.107075 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:23.107045 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rr7gh" Jul 15 18:59:23.146964 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:23.146929 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mf5q5"] Jul 15 18:59:23.149274 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:59:23.149245 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb21b0c06_28df_48dc_9dfd_34acda9908e4.slice/crio-115c7d689528207d131ab1657caef53657d9ef0ddcc732bb60355a5bd40625d6 WatchSource:0}: Error finding container 115c7d689528207d131ab1657caef53657d9ef0ddcc732bb60355a5bd40625d6: Status 404 returned error can't find the container with id 115c7d689528207d131ab1657caef53657d9ef0ddcc732bb60355a5bd40625d6 Jul 15 18:59:23.244893 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:23.244867 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rr7gh"] Jul 15 18:59:23.246565 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:59:23.246538 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03bb70c7_4249_433a_8a70_b582613aa5c8.slice/crio-85d083c4468fe56f4842dc3c9c6676c06ccbeb0f0da98ddba4031281d34455d8 WatchSource:0}: Error finding container 85d083c4468fe56f4842dc3c9c6676c06ccbeb0f0da98ddba4031281d34455d8: Status 404 returned error can't find the container with id 85d083c4468fe56f4842dc3c9c6676c06ccbeb0f0da98ddba4031281d34455d8 Jul 15 18:59:23.427050 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:23.427010 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rr7gh" event={"ID":"03bb70c7-4249-433a-8a70-b582613aa5c8","Type":"ContainerStarted","Data":"85d083c4468fe56f4842dc3c9c6676c06ccbeb0f0da98ddba4031281d34455d8"} Jul 15 18:59:23.428127 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:23.428105 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mf5q5" event={"ID":"b21b0c06-28df-48dc-9dfd-34acda9908e4","Type":"ContainerStarted","Data":"115c7d689528207d131ab1657caef53657d9ef0ddcc732bb60355a5bd40625d6"} Jul 15 18:59:27.455177 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:27.455143 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mf5q5" event={"ID":"b21b0c06-28df-48dc-9dfd-34acda9908e4","Type":"ContainerStarted","Data":"18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739"} Jul 15 18:59:27.456634 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:27.456611 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rr7gh" event={"ID":"03bb70c7-4249-433a-8a70-b582613aa5c8","Type":"ContainerStarted","Data":"afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c"} Jul 15 18:59:27.470249 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:27.470204 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mf5q5" podStartSLOduration=1.8018539310000001 podStartE2EDuration="5.470188201s" totalImagesPullingTime="3.66833427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:59:27.469332981 +0000 UTC m=+517.817045280" watchObservedRunningTime="2026-07-15 18:59:27.470188201 +0000 UTC m=+517.817900500" Jul 15 18:59:27.483908 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:27.483866 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-rr7gh" podStartSLOduration=1.923336531 podStartE2EDuration="5.483851557s" totalImagesPullingTime="3.560515026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:59:27.482689347 +0000 UTC m=+517.830401648" watchObservedRunningTime="2026-07-15 18:59:27.483851557 +0000 UTC m=+517.831563855" Jul 15 18:59:27.508367 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:27.508338 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mf5q5"] Jul 15 18:59:29.465892 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:29.465859 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mf5q5" podUID="b21b0c06-28df-48dc-9dfd-34acda9908e4" containerName="authorino" containerID="cri-o://18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739" gracePeriod=30 Jul 15 18:59:29.713530 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:29.713474 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mf5q5" Jul 15 18:59:29.724439 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:29.724411 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8qnz\" (UniqueName: \"kubernetes.io/projected/b21b0c06-28df-48dc-9dfd-34acda9908e4-kube-api-access-s8qnz\") pod \"b21b0c06-28df-48dc-9dfd-34acda9908e4\" (UID: \"b21b0c06-28df-48dc-9dfd-34acda9908e4\") " Jul 15 18:59:29.726228 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:29.726202 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b21b0c06-28df-48dc-9dfd-34acda9908e4-kube-api-access-s8qnz" (OuterVolumeSpecName: "kube-api-access-s8qnz") pod "b21b0c06-28df-48dc-9dfd-34acda9908e4" (UID: "b21b0c06-28df-48dc-9dfd-34acda9908e4"). InnerVolumeSpecName "kube-api-access-s8qnz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:59:29.824883 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:29.824850 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8qnz\" (UniqueName: \"kubernetes.io/projected/b21b0c06-28df-48dc-9dfd-34acda9908e4-kube-api-access-s8qnz\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:59:30.471882 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:30.471794 2574 generic.go:356] "Generic (PLEG): container finished" podID="b21b0c06-28df-48dc-9dfd-34acda9908e4" containerID="18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739" exitCode=0 Jul 15 18:59:30.471882 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:30.471857 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mf5q5" Jul 15 18:59:30.471882 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:30.471874 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mf5q5" event={"ID":"b21b0c06-28df-48dc-9dfd-34acda9908e4","Type":"ContainerDied","Data":"18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739"} Jul 15 18:59:30.472378 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:30.471914 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mf5q5" event={"ID":"b21b0c06-28df-48dc-9dfd-34acda9908e4","Type":"ContainerDied","Data":"115c7d689528207d131ab1657caef53657d9ef0ddcc732bb60355a5bd40625d6"} Jul 15 18:59:30.472378 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:30.471931 2574 scope.go:117] "RemoveContainer" containerID="18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739" Jul 15 18:59:30.480206 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:30.480191 2574 scope.go:117] "RemoveContainer" containerID="18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739" Jul 15 18:59:30.480484 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:59:30.480461 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739\": container with ID starting with 18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739 not found: ID does not exist" containerID="18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739" Jul 15 18:59:30.480572 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:30.480494 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739"} err="failed to get container status \"18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739\": rpc error: code = NotFound desc = could not find container \"18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739\": container with ID starting with 18529e5b327970d7d7b7ffc9d3796511c2dcf1b3d2b15df12c0d0bff80df2739 not found: ID does not exist" Jul 15 18:59:30.488374 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:30.488354 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mf5q5"] Jul 15 18:59:30.491768 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:30.491744 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mf5q5"] Jul 15 18:59:32.205900 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:32.205870 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b21b0c06-28df-48dc-9dfd-34acda9908e4" path="/var/lib/kubelet/pods/b21b0c06-28df-48dc-9dfd-34acda9908e4/volumes" Jul 15 18:59:40.576214 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:40.576171 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:40.683618 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:40.683584 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:41.188016 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:41.187983 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:52.843376 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:52.843336 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:53.547517 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:53.547484 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:53.794432 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:53.794404 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fvxnn"] Jul 15 18:59:53.794714 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:53.794703 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b21b0c06-28df-48dc-9dfd-34acda9908e4" containerName="authorino" Jul 15 18:59:53.794756 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:53.794715 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b21b0c06-28df-48dc-9dfd-34acda9908e4" containerName="authorino" Jul 15 18:59:53.794805 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:53.794777 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b21b0c06-28df-48dc-9dfd-34acda9908e4" containerName="authorino" Jul 15 18:59:53.798947 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:53.798908 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fvxnn" Jul 15 18:59:53.801841 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:53.801818 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fvxnn"] Jul 15 18:59:53.908287 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:53.908255 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlc4r\" (UniqueName: \"kubernetes.io/projected/34ae5214-1271-4317-a90f-3a5c61331982-kube-api-access-rlc4r\") pod \"authorino-5d4b67c446-fvxnn\" (UID: \"34ae5214-1271-4317-a90f-3a5c61331982\") " pod="kuadrant-system/authorino-5d4b67c446-fvxnn" Jul 15 18:59:54.009673 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.009644 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlc4r\" (UniqueName: \"kubernetes.io/projected/34ae5214-1271-4317-a90f-3a5c61331982-kube-api-access-rlc4r\") pod \"authorino-5d4b67c446-fvxnn\" (UID: \"34ae5214-1271-4317-a90f-3a5c61331982\") " pod="kuadrant-system/authorino-5d4b67c446-fvxnn" Jul 15 18:59:54.017867 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.017845 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlc4r\" (UniqueName: \"kubernetes.io/projected/34ae5214-1271-4317-a90f-3a5c61331982-kube-api-access-rlc4r\") pod \"authorino-5d4b67c446-fvxnn\" (UID: \"34ae5214-1271-4317-a90f-3a5c61331982\") " pod="kuadrant-system/authorino-5d4b67c446-fvxnn" Jul 15 18:59:54.044814 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.044778 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fvxnn"] Jul 15 18:59:54.045013 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.045002 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fvxnn" Jul 15 18:59:54.072832 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.072806 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c45cf6648-7vhbg"] Jul 15 18:59:54.076435 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.076417 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c45cf6648-7vhbg" Jul 15 18:59:54.081346 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.081321 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c45cf6648-7vhbg"] Jul 15 18:59:54.097397 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.097358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 18:59:54.110172 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.110108 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9szh\" (UniqueName: \"kubernetes.io/projected/c0a17d4e-3425-402e-98c1-be2b3e2d0591-kube-api-access-z9szh\") pod \"authorino-7c45cf6648-7vhbg\" (UID: \"c0a17d4e-3425-402e-98c1-be2b3e2d0591\") " pod="kuadrant-system/authorino-7c45cf6648-7vhbg" Jul 15 18:59:54.175084 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.175062 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fvxnn"] Jul 15 18:59:54.177089 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:59:54.177064 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34ae5214_1271_4317_a90f_3a5c61331982.slice/crio-20bc7ce436d9f2aaea8c547e1a816c63f9d1b9e66af8cfdd839540a6770fa306 WatchSource:0}: Error finding container 20bc7ce436d9f2aaea8c547e1a816c63f9d1b9e66af8cfdd839540a6770fa306: Status 404 returned error can't find the container with id 20bc7ce436d9f2aaea8c547e1a816c63f9d1b9e66af8cfdd839540a6770fa306 Jul 15 18:59:54.211069 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.211043 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9szh\" (UniqueName: \"kubernetes.io/projected/c0a17d4e-3425-402e-98c1-be2b3e2d0591-kube-api-access-z9szh\") pod \"authorino-7c45cf6648-7vhbg\" (UID: \"c0a17d4e-3425-402e-98c1-be2b3e2d0591\") " pod="kuadrant-system/authorino-7c45cf6648-7vhbg" Jul 15 18:59:54.220415 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.220392 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9szh\" (UniqueName: \"kubernetes.io/projected/c0a17d4e-3425-402e-98c1-be2b3e2d0591-kube-api-access-z9szh\") pod \"authorino-7c45cf6648-7vhbg\" (UID: \"c0a17d4e-3425-402e-98c1-be2b3e2d0591\") " pod="kuadrant-system/authorino-7c45cf6648-7vhbg" Jul 15 18:59:54.371156 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.371074 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c45cf6648-7vhbg"] Jul 15 18:59:54.375809 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.371802 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c45cf6648-7vhbg" Jul 15 18:59:54.403609 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.403557 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d4c5f7b88-tndsb"] Jul 15 18:59:54.406852 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.406829 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" Jul 15 18:59:54.409047 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.409027 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 15 18:59:54.415465 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.415182 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d4c5f7b88-tndsb"] Jul 15 18:59:54.513777 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.513748 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9snf\" (UniqueName: \"kubernetes.io/projected/4321c481-c77f-42c3-b396-e260a61c20ff-kube-api-access-m9snf\") pod \"authorino-7d4c5f7b88-tndsb\" (UID: \"4321c481-c77f-42c3-b396-e260a61c20ff\") " pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" Jul 15 18:59:54.513908 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.513803 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4321c481-c77f-42c3-b396-e260a61c20ff-tls-cert\") pod \"authorino-7d4c5f7b88-tndsb\" (UID: \"4321c481-c77f-42c3-b396-e260a61c20ff\") " pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" Jul 15 18:59:54.518844 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.518823 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c45cf6648-7vhbg"] Jul 15 18:59:54.520499 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:59:54.520478 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0a17d4e_3425_402e_98c1_be2b3e2d0591.slice/crio-ebd145770ca4c60b60635efed76711dea63f852e633fb839b63515d829f58199 WatchSource:0}: Error finding container ebd145770ca4c60b60635efed76711dea63f852e633fb839b63515d829f58199: Status 404 returned error can't find the container with id ebd145770ca4c60b60635efed76711dea63f852e633fb839b63515d829f58199 Jul 15 18:59:54.587282 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.587257 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c45cf6648-7vhbg" event={"ID":"c0a17d4e-3425-402e-98c1-be2b3e2d0591","Type":"ContainerStarted","Data":"ebd145770ca4c60b60635efed76711dea63f852e633fb839b63515d829f58199"} Jul 15 18:59:54.588277 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.588255 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fvxnn" event={"ID":"34ae5214-1271-4317-a90f-3a5c61331982","Type":"ContainerStarted","Data":"20bc7ce436d9f2aaea8c547e1a816c63f9d1b9e66af8cfdd839540a6770fa306"} Jul 15 18:59:54.614339 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.614315 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9snf\" (UniqueName: \"kubernetes.io/projected/4321c481-c77f-42c3-b396-e260a61c20ff-kube-api-access-m9snf\") pod \"authorino-7d4c5f7b88-tndsb\" (UID: \"4321c481-c77f-42c3-b396-e260a61c20ff\") " pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" Jul 15 18:59:54.614423 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.614348 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4321c481-c77f-42c3-b396-e260a61c20ff-tls-cert\") pod \"authorino-7d4c5f7b88-tndsb\" (UID: \"4321c481-c77f-42c3-b396-e260a61c20ff\") " pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" Jul 15 18:59:54.616567 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.616547 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4321c481-c77f-42c3-b396-e260a61c20ff-tls-cert\") pod \"authorino-7d4c5f7b88-tndsb\" (UID: \"4321c481-c77f-42c3-b396-e260a61c20ff\") " pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" Jul 15 18:59:54.624138 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.624086 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9snf\" (UniqueName: \"kubernetes.io/projected/4321c481-c77f-42c3-b396-e260a61c20ff-kube-api-access-m9snf\") pod \"authorino-7d4c5f7b88-tndsb\" (UID: \"4321c481-c77f-42c3-b396-e260a61c20ff\") " pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" Jul 15 18:59:54.731880 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.731846 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" Jul 15 18:59:54.856007 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:54.855984 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d4c5f7b88-tndsb"] Jul 15 18:59:54.857461 ip-10-0-137-11 kubenswrapper[2574]: W0715 18:59:54.857433 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4321c481_c77f_42c3_b396_e260a61c20ff.slice/crio-1e225a3ebae308228e5521bdf91875f8b0f6fdc168eb4fb441ab36c867bc098f WatchSource:0}: Error finding container 1e225a3ebae308228e5521bdf91875f8b0f6fdc168eb4fb441ab36c867bc098f: Status 404 returned error can't find the container with id 1e225a3ebae308228e5521bdf91875f8b0f6fdc168eb4fb441ab36c867bc098f Jul 15 18:59:55.594577 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.594541 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fvxnn" event={"ID":"34ae5214-1271-4317-a90f-3a5c61331982","Type":"ContainerStarted","Data":"31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b"} Jul 15 18:59:55.595020 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.594579 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-fvxnn" podUID="34ae5214-1271-4317-a90f-3a5c61331982" containerName="authorino" containerID="cri-o://31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b" gracePeriod=30 Jul 15 18:59:55.595722 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.595697 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" event={"ID":"4321c481-c77f-42c3-b396-e260a61c20ff","Type":"ContainerStarted","Data":"1e225a3ebae308228e5521bdf91875f8b0f6fdc168eb4fb441ab36c867bc098f"} Jul 15 18:59:55.597056 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.597034 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c45cf6648-7vhbg" event={"ID":"c0a17d4e-3425-402e-98c1-be2b3e2d0591","Type":"ContainerStarted","Data":"6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd"} Jul 15 18:59:55.597143 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.597089 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7c45cf6648-7vhbg" podUID="c0a17d4e-3425-402e-98c1-be2b3e2d0591" containerName="authorino" containerID="cri-o://6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd" gracePeriod=30 Jul 15 18:59:55.609942 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.609901 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-fvxnn" podStartSLOduration=2.223722053 podStartE2EDuration="2.609889305s" totalImagesPullingTime="386.167252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:59:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:59:55.607259789 +0000 UTC m=+545.954972100" watchObservedRunningTime="2026-07-15 18:59:55.609889305 +0000 UTC m=+545.957601603" Jul 15 18:59:55.621889 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.621850 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c45cf6648-7vhbg" podStartSLOduration=1.092727073 podStartE2EDuration="1.621835811s" totalImagesPullingTime="529.108738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:59:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:59:55.620632204 +0000 UTC m=+545.968344505" watchObservedRunningTime="2026-07-15 18:59:55.621835811 +0000 UTC m=+545.969548113" Jul 15 18:59:55.847229 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.847173 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fvxnn" Jul 15 18:59:55.856131 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.856108 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c45cf6648-7vhbg" Jul 15 18:59:55.925283 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.925260 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9szh\" (UniqueName: \"kubernetes.io/projected/c0a17d4e-3425-402e-98c1-be2b3e2d0591-kube-api-access-z9szh\") pod \"c0a17d4e-3425-402e-98c1-be2b3e2d0591\" (UID: \"c0a17d4e-3425-402e-98c1-be2b3e2d0591\") " Jul 15 18:59:55.925418 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.925316 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlc4r\" (UniqueName: \"kubernetes.io/projected/34ae5214-1271-4317-a90f-3a5c61331982-kube-api-access-rlc4r\") pod \"34ae5214-1271-4317-a90f-3a5c61331982\" (UID: \"34ae5214-1271-4317-a90f-3a5c61331982\") " Jul 15 18:59:55.927300 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.927268 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0a17d4e-3425-402e-98c1-be2b3e2d0591-kube-api-access-z9szh" (OuterVolumeSpecName: "kube-api-access-z9szh") pod "c0a17d4e-3425-402e-98c1-be2b3e2d0591" (UID: "c0a17d4e-3425-402e-98c1-be2b3e2d0591"). InnerVolumeSpecName "kube-api-access-z9szh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:59:55.927409 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:55.927391 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34ae5214-1271-4317-a90f-3a5c61331982-kube-api-access-rlc4r" (OuterVolumeSpecName: "kube-api-access-rlc4r") pod "34ae5214-1271-4317-a90f-3a5c61331982" (UID: "34ae5214-1271-4317-a90f-3a5c61331982"). InnerVolumeSpecName "kube-api-access-rlc4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:59:56.025922 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.025895 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlc4r\" (UniqueName: \"kubernetes.io/projected/34ae5214-1271-4317-a90f-3a5c61331982-kube-api-access-rlc4r\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:59:56.025922 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.025924 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9szh\" (UniqueName: \"kubernetes.io/projected/c0a17d4e-3425-402e-98c1-be2b3e2d0591-kube-api-access-z9szh\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:59:56.604048 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.604016 2574 generic.go:356] "Generic (PLEG): container finished" podID="34ae5214-1271-4317-a90f-3a5c61331982" containerID="31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b" exitCode=0 Jul 15 18:59:56.604487 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.604064 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fvxnn" Jul 15 18:59:56.604487 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.604103 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fvxnn" event={"ID":"34ae5214-1271-4317-a90f-3a5c61331982","Type":"ContainerDied","Data":"31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b"} Jul 15 18:59:56.604487 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.604149 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fvxnn" event={"ID":"34ae5214-1271-4317-a90f-3a5c61331982","Type":"ContainerDied","Data":"20bc7ce436d9f2aaea8c547e1a816c63f9d1b9e66af8cfdd839540a6770fa306"} Jul 15 18:59:56.604487 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.604175 2574 scope.go:117] "RemoveContainer" containerID="31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b" Jul 15 18:59:56.606036 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.605815 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" event={"ID":"4321c481-c77f-42c3-b396-e260a61c20ff","Type":"ContainerStarted","Data":"ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1"} Jul 15 18:59:56.607088 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.607069 2574 generic.go:356] "Generic (PLEG): container finished" podID="c0a17d4e-3425-402e-98c1-be2b3e2d0591" containerID="6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd" exitCode=0 Jul 15 18:59:56.607193 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.607097 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c45cf6648-7vhbg" Jul 15 18:59:56.607193 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.607147 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c45cf6648-7vhbg" event={"ID":"c0a17d4e-3425-402e-98c1-be2b3e2d0591","Type":"ContainerDied","Data":"6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd"} Jul 15 18:59:56.607193 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.607183 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c45cf6648-7vhbg" event={"ID":"c0a17d4e-3425-402e-98c1-be2b3e2d0591","Type":"ContainerDied","Data":"ebd145770ca4c60b60635efed76711dea63f852e633fb839b63515d829f58199"} Jul 15 18:59:56.613078 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.613062 2574 scope.go:117] "RemoveContainer" containerID="31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b" Jul 15 18:59:56.613303 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:59:56.613287 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b\": container with ID starting with 31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b not found: ID does not exist" containerID="31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b" Jul 15 18:59:56.613340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.613312 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b"} err="failed to get container status \"31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b\": rpc error: code = NotFound desc = could not find container \"31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b\": container with ID starting with 31d1db247bf7bbb271f676cea4890a7bf6880e7c7e39788ef75fae1490f31f2b not found: ID does not exist" Jul 15 18:59:56.613340 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.613327 2574 scope.go:117] "RemoveContainer" containerID="6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd" Jul 15 18:59:56.620551 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.620535 2574 scope.go:117] "RemoveContainer" containerID="6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd" Jul 15 18:59:56.620855 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:59:56.620801 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd\": container with ID starting with 6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd not found: ID does not exist" containerID="6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd" Jul 15 18:59:56.620855 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.620824 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd"} err="failed to get container status \"6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd\": rpc error: code = NotFound desc = could not find container \"6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd\": container with ID starting with 6b5b96464502959642b178e14a7eaee77bb337d3e3d20bbfcd4f45a980ab64fd not found: ID does not exist" Jul 15 18:59:56.622951 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.622916 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" podStartSLOduration=1.637708683 podStartE2EDuration="2.622903432s" totalImagesPullingTime="985.194749ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:59:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:59:56.621403956 +0000 UTC m=+546.969116256" watchObservedRunningTime="2026-07-15 18:59:56.622903432 +0000 UTC m=+546.970615731" Jul 15 18:59:56.637454 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.637432 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fvxnn"] Jul 15 18:59:56.643953 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.643931 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fvxnn"] Jul 15 18:59:56.650444 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.650418 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rr7gh"] Jul 15 18:59:56.650592 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.650571 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-rr7gh" podUID="03bb70c7-4249-433a-8a70-b582613aa5c8" containerName="authorino" containerID="cri-o://afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c" gracePeriod=30 Jul 15 18:59:56.657622 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.657596 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c45cf6648-7vhbg"] Jul 15 18:59:56.661077 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.661054 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7c45cf6648-7vhbg"] Jul 15 18:59:56.894604 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.894581 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rr7gh" Jul 15 18:59:56.934029 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.934004 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4xr9\" (UniqueName: \"kubernetes.io/projected/03bb70c7-4249-433a-8a70-b582613aa5c8-kube-api-access-q4xr9\") pod \"03bb70c7-4249-433a-8a70-b582613aa5c8\" (UID: \"03bb70c7-4249-433a-8a70-b582613aa5c8\") " Jul 15 18:59:56.935991 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:56.935961 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03bb70c7-4249-433a-8a70-b582613aa5c8-kube-api-access-q4xr9" (OuterVolumeSpecName: "kube-api-access-q4xr9") pod "03bb70c7-4249-433a-8a70-b582613aa5c8" (UID: "03bb70c7-4249-433a-8a70-b582613aa5c8"). InnerVolumeSpecName "kube-api-access-q4xr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:59:57.035233 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:57.035207 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4xr9\" (UniqueName: \"kubernetes.io/projected/03bb70c7-4249-433a-8a70-b582613aa5c8-kube-api-access-q4xr9\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 18:59:57.615122 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:57.615089 2574 generic.go:356] "Generic (PLEG): container finished" podID="03bb70c7-4249-433a-8a70-b582613aa5c8" containerID="afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c" exitCode=0 Jul 15 18:59:57.615577 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:57.615135 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rr7gh" Jul 15 18:59:57.615577 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:57.615132 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rr7gh" event={"ID":"03bb70c7-4249-433a-8a70-b582613aa5c8","Type":"ContainerDied","Data":"afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c"} Jul 15 18:59:57.615577 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:57.615173 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rr7gh" event={"ID":"03bb70c7-4249-433a-8a70-b582613aa5c8","Type":"ContainerDied","Data":"85d083c4468fe56f4842dc3c9c6676c06ccbeb0f0da98ddba4031281d34455d8"} Jul 15 18:59:57.615577 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:57.615194 2574 scope.go:117] "RemoveContainer" containerID="afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c" Jul 15 18:59:57.624448 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:57.624408 2574 scope.go:117] "RemoveContainer" containerID="afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c" Jul 15 18:59:57.624657 ip-10-0-137-11 kubenswrapper[2574]: E0715 18:59:57.624638 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c\": container with ID starting with afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c not found: ID does not exist" containerID="afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c" Jul 15 18:59:57.624714 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:57.624669 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c"} err="failed to get container status \"afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c\": rpc error: code = NotFound desc = could not find container \"afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c\": container with ID starting with afa3e21652a62ce26c04b596cbcc1fc24d434d25687fbd3e420359533a70fb8c not found: ID does not exist" Jul 15 18:59:57.636044 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:57.636021 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rr7gh"] Jul 15 18:59:57.639973 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:57.639954 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rr7gh"] Jul 15 18:59:58.206427 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:58.206393 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03bb70c7-4249-433a-8a70-b582613aa5c8" path="/var/lib/kubelet/pods/03bb70c7-4249-433a-8a70-b582613aa5c8/volumes" Jul 15 18:59:58.206707 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:58.206695 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34ae5214-1271-4317-a90f-3a5c61331982" path="/var/lib/kubelet/pods/34ae5214-1271-4317-a90f-3a5c61331982/volumes" Jul 15 18:59:58.207000 ip-10-0-137-11 kubenswrapper[2574]: I0715 18:59:58.206987 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0a17d4e-3425-402e-98c1-be2b3e2d0591" path="/var/lib/kubelet/pods/c0a17d4e-3425-402e-98c1-be2b3e2d0591/volumes" Jul 15 19:00:01.831692 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.831654 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6459597c95-l7hgg"] Jul 15 19:00:01.832247 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.832130 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0a17d4e-3425-402e-98c1-be2b3e2d0591" containerName="authorino" Jul 15 19:00:01.832247 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.832151 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0a17d4e-3425-402e-98c1-be2b3e2d0591" containerName="authorino" Jul 15 19:00:01.832247 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.832170 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03bb70c7-4249-433a-8a70-b582613aa5c8" containerName="authorino" Jul 15 19:00:01.832247 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.832178 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="03bb70c7-4249-433a-8a70-b582613aa5c8" containerName="authorino" Jul 15 19:00:01.832247 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.832192 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34ae5214-1271-4317-a90f-3a5c61331982" containerName="authorino" Jul 15 19:00:01.832247 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.832202 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="34ae5214-1271-4317-a90f-3a5c61331982" containerName="authorino" Jul 15 19:00:01.832572 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.832280 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="34ae5214-1271-4317-a90f-3a5c61331982" containerName="authorino" Jul 15 19:00:01.832572 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.832297 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="03bb70c7-4249-433a-8a70-b582613aa5c8" containerName="authorino" Jul 15 19:00:01.832572 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.832307 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0a17d4e-3425-402e-98c1-be2b3e2d0591" containerName="authorino" Jul 15 19:00:01.838796 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.838760 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:01.841483 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.841459 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qqrkm\"" Jul 15 19:00:01.841603 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.841481 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 19:00:01.841733 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.841715 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 19:00:01.848002 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.847978 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6459597c95-l7hgg"] Jul 15 19:00:01.849285 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.849225 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:01.950886 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.950845 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:01.976197 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.976171 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6x7b\" (UniqueName: \"kubernetes.io/projected/aa321c27-8e97-456c-b99a-a68a87dd5edc-kube-api-access-q6x7b\") pod \"maas-controller-6459597c95-l7hgg\" (UID: \"aa321c27-8e97-456c-b99a-a68a87dd5edc\") " pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:01.976316 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:01.976204 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa321c27-8e97-456c-b99a-a68a87dd5edc-webhook-cert\") pod \"maas-controller-6459597c95-l7hgg\" (UID: \"aa321c27-8e97-456c-b99a-a68a87dd5edc\") " pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:02.016694 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.016668 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7dcfcc7558-hwx45"] Jul 15 19:00:02.020098 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.020082 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:00:02.030610 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.030589 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dcfcc7558-hwx45"] Jul 15 19:00:02.077217 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.077185 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq5kd\" (UniqueName: \"kubernetes.io/projected/d48a04c9-9e35-49b4-a9b5-694d97dc1043-kube-api-access-vq5kd\") pod \"maas-controller-7dcfcc7558-hwx45\" (UID: \"d48a04c9-9e35-49b4-a9b5-694d97dc1043\") " pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:00:02.077365 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.077245 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6x7b\" (UniqueName: \"kubernetes.io/projected/aa321c27-8e97-456c-b99a-a68a87dd5edc-kube-api-access-q6x7b\") pod \"maas-controller-6459597c95-l7hgg\" (UID: \"aa321c27-8e97-456c-b99a-a68a87dd5edc\") " pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:02.077365 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.077279 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa321c27-8e97-456c-b99a-a68a87dd5edc-webhook-cert\") pod \"maas-controller-6459597c95-l7hgg\" (UID: \"aa321c27-8e97-456c-b99a-a68a87dd5edc\") " pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:02.077365 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.077300 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d48a04c9-9e35-49b4-a9b5-694d97dc1043-webhook-cert\") pod \"maas-controller-7dcfcc7558-hwx45\" (UID: \"d48a04c9-9e35-49b4-a9b5-694d97dc1043\") " pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:00:02.077533 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:00:02.077395 2574 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 15 19:00:02.077533 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:00:02.077473 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa321c27-8e97-456c-b99a-a68a87dd5edc-webhook-cert podName:aa321c27-8e97-456c-b99a-a68a87dd5edc nodeName:}" failed. No retries permitted until 2026-07-15 19:00:02.57745109 +0000 UTC m=+552.925163380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/aa321c27-8e97-456c-b99a-a68a87dd5edc-webhook-cert") pod "maas-controller-6459597c95-l7hgg" (UID: "aa321c27-8e97-456c-b99a-a68a87dd5edc") : secret "maas-controller-webhook-cert" not found Jul 15 19:00:02.085452 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.085400 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6x7b\" (UniqueName: \"kubernetes.io/projected/aa321c27-8e97-456c-b99a-a68a87dd5edc-kube-api-access-q6x7b\") pod \"maas-controller-6459597c95-l7hgg\" (UID: \"aa321c27-8e97-456c-b99a-a68a87dd5edc\") " pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:02.178519 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.178484 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq5kd\" (UniqueName: \"kubernetes.io/projected/d48a04c9-9e35-49b4-a9b5-694d97dc1043-kube-api-access-vq5kd\") pod \"maas-controller-7dcfcc7558-hwx45\" (UID: \"d48a04c9-9e35-49b4-a9b5-694d97dc1043\") " pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:00:02.178670 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.178545 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d48a04c9-9e35-49b4-a9b5-694d97dc1043-webhook-cert\") pod \"maas-controller-7dcfcc7558-hwx45\" (UID: \"d48a04c9-9e35-49b4-a9b5-694d97dc1043\") " pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:00:02.178670 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:00:02.178642 2574 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 15 19:00:02.178761 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:00:02.178689 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d48a04c9-9e35-49b4-a9b5-694d97dc1043-webhook-cert podName:d48a04c9-9e35-49b4-a9b5-694d97dc1043 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:02.678676959 +0000 UTC m=+553.026389236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d48a04c9-9e35-49b4-a9b5-694d97dc1043-webhook-cert") pod "maas-controller-7dcfcc7558-hwx45" (UID: "d48a04c9-9e35-49b4-a9b5-694d97dc1043") : secret "maas-controller-webhook-cert" not found Jul 15 19:00:02.187424 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.187400 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq5kd\" (UniqueName: \"kubernetes.io/projected/d48a04c9-9e35-49b4-a9b5-694d97dc1043-kube-api-access-vq5kd\") pod \"maas-controller-7dcfcc7558-hwx45\" (UID: \"d48a04c9-9e35-49b4-a9b5-694d97dc1043\") " pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:00:02.275983 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.275951 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:02.581942 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.581909 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa321c27-8e97-456c-b99a-a68a87dd5edc-webhook-cert\") pod \"maas-controller-6459597c95-l7hgg\" (UID: \"aa321c27-8e97-456c-b99a-a68a87dd5edc\") " pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:02.584254 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.584233 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa321c27-8e97-456c-b99a-a68a87dd5edc-webhook-cert\") pod \"maas-controller-6459597c95-l7hgg\" (UID: \"aa321c27-8e97-456c-b99a-a68a87dd5edc\") " pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:02.682876 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.682846 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d48a04c9-9e35-49b4-a9b5-694d97dc1043-webhook-cert\") pod \"maas-controller-7dcfcc7558-hwx45\" (UID: \"d48a04c9-9e35-49b4-a9b5-694d97dc1043\") " pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:00:02.685100 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.685079 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d48a04c9-9e35-49b4-a9b5-694d97dc1043-webhook-cert\") pod \"maas-controller-7dcfcc7558-hwx45\" (UID: \"d48a04c9-9e35-49b4-a9b5-694d97dc1043\") " pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:00:02.750449 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.750404 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:02.883853 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.883805 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6459597c95-l7hgg"] Jul 15 19:00:02.886646 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:00:02.886620 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa321c27_8e97_456c_b99a_a68a87dd5edc.slice/crio-5c3865850639a0bfd93d00313d3e4478c9cd62430e573f6142773bc22d934e12 WatchSource:0}: Error finding container 5c3865850639a0bfd93d00313d3e4478c9cd62430e573f6142773bc22d934e12: Status 404 returned error can't find the container with id 5c3865850639a0bfd93d00313d3e4478c9cd62430e573f6142773bc22d934e12 Jul 15 19:00:02.930884 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:02.930850 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:00:03.056115 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:03.056088 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dcfcc7558-hwx45"] Jul 15 19:00:03.057624 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:00:03.057601 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd48a04c9_9e35_49b4_a9b5_694d97dc1043.slice/crio-554f471cc7fd31ff2c95b1bf0f4bb023f5a85328e995f0b5023d6ec0022e0654 WatchSource:0}: Error finding container 554f471cc7fd31ff2c95b1bf0f4bb023f5a85328e995f0b5023d6ec0022e0654: Status 404 returned error can't find the container with id 554f471cc7fd31ff2c95b1bf0f4bb023f5a85328e995f0b5023d6ec0022e0654 Jul 15 19:00:03.577818 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:03.577769 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:03.644546 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:03.644514 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" event={"ID":"d48a04c9-9e35-49b4-a9b5-694d97dc1043","Type":"ContainerStarted","Data":"554f471cc7fd31ff2c95b1bf0f4bb023f5a85328e995f0b5023d6ec0022e0654"} Jul 15 19:00:03.645555 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:03.645527 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6459597c95-l7hgg" event={"ID":"aa321c27-8e97-456c-b99a-a68a87dd5edc","Type":"ContainerStarted","Data":"5c3865850639a0bfd93d00313d3e4478c9cd62430e573f6142773bc22d934e12"} Jul 15 19:00:06.367799 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:06.367756 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:06.466933 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:06.466897 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:06.663143 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:06.663043 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" event={"ID":"d48a04c9-9e35-49b4-a9b5-694d97dc1043","Type":"ContainerStarted","Data":"1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546"} Jul 15 19:00:06.663143 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:06.663107 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:00:06.664460 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:06.664436 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6459597c95-l7hgg" event={"ID":"aa321c27-8e97-456c-b99a-a68a87dd5edc","Type":"ContainerStarted","Data":"a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab"} Jul 15 19:00:06.664582 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:06.664565 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:06.681808 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:06.681746 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" podStartSLOduration=1.929024362 podStartE2EDuration="4.6817315s" totalImagesPullingTime="2.752707138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:06.68029731 +0000 UTC m=+557.028009631" watchObservedRunningTime="2026-07-15 19:00:06.6817315 +0000 UTC m=+557.029443799" Jul 15 19:00:06.697687 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:06.697642 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6459597c95-l7hgg" podStartSLOduration=2.7632484379999998 podStartE2EDuration="5.697629146s" totalImagesPullingTime="2.934380708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:06.695959622 +0000 UTC m=+557.043671938" watchObservedRunningTime="2026-07-15 19:00:06.697629146 +0000 UTC m=+557.045341479" Jul 15 19:00:17.673090 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:17.673058 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:00:17.673523 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:17.673125 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:17.741898 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:17.741869 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6459597c95-l7hgg"] Jul 15 19:00:17.742069 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:17.742039 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6459597c95-l7hgg" podUID="aa321c27-8e97-456c-b99a-a68a87dd5edc" containerName="manager" containerID="cri-o://a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab" gracePeriod=30 Jul 15 19:00:17.984965 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:17.984938 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:18.107846 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.107813 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa321c27-8e97-456c-b99a-a68a87dd5edc-webhook-cert\") pod \"aa321c27-8e97-456c-b99a-a68a87dd5edc\" (UID: \"aa321c27-8e97-456c-b99a-a68a87dd5edc\") " Jul 15 19:00:18.107846 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.107849 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6x7b\" (UniqueName: \"kubernetes.io/projected/aa321c27-8e97-456c-b99a-a68a87dd5edc-kube-api-access-q6x7b\") pod \"aa321c27-8e97-456c-b99a-a68a87dd5edc\" (UID: \"aa321c27-8e97-456c-b99a-a68a87dd5edc\") " Jul 15 19:00:18.109969 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.109936 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa321c27-8e97-456c-b99a-a68a87dd5edc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "aa321c27-8e97-456c-b99a-a68a87dd5edc" (UID: "aa321c27-8e97-456c-b99a-a68a87dd5edc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:00:18.110078 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.110048 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa321c27-8e97-456c-b99a-a68a87dd5edc-kube-api-access-q6x7b" (OuterVolumeSpecName: "kube-api-access-q6x7b") pod "aa321c27-8e97-456c-b99a-a68a87dd5edc" (UID: "aa321c27-8e97-456c-b99a-a68a87dd5edc"). InnerVolumeSpecName "kube-api-access-q6x7b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:00:18.208379 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.208348 2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa321c27-8e97-456c-b99a-a68a87dd5edc-webhook-cert\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:00:18.208379 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.208375 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6x7b\" (UniqueName: \"kubernetes.io/projected/aa321c27-8e97-456c-b99a-a68a87dd5edc-kube-api-access-q6x7b\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:00:18.724725 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.724688 2574 generic.go:356] "Generic (PLEG): container finished" podID="aa321c27-8e97-456c-b99a-a68a87dd5edc" containerID="a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab" exitCode=0 Jul 15 19:00:18.725126 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.724737 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6459597c95-l7hgg" Jul 15 19:00:18.725126 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.724773 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6459597c95-l7hgg" event={"ID":"aa321c27-8e97-456c-b99a-a68a87dd5edc","Type":"ContainerDied","Data":"a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab"} Jul 15 19:00:18.725126 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.724842 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6459597c95-l7hgg" event={"ID":"aa321c27-8e97-456c-b99a-a68a87dd5edc","Type":"ContainerDied","Data":"5c3865850639a0bfd93d00313d3e4478c9cd62430e573f6142773bc22d934e12"} Jul 15 19:00:18.725126 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.724864 2574 scope.go:117] "RemoveContainer" containerID="a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab" Jul 15 19:00:18.733149 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.733129 2574 scope.go:117] "RemoveContainer" containerID="a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab" Jul 15 19:00:18.733386 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:00:18.733365 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab\": container with ID starting with a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab not found: ID does not exist" containerID="a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab" Jul 15 19:00:18.733431 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.733395 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab"} err="failed to get container status \"a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab\": rpc error: code = NotFound desc = could not find container \"a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab\": container with ID starting with a9fb8f03c946a0666419ec666c7e186eb064f150687f188453d371c00f68e4ab not found: ID does not exist" Jul 15 19:00:18.740865 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.740844 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6459597c95-l7hgg"] Jul 15 19:00:18.744285 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:18.744260 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6459597c95-l7hgg"] Jul 15 19:00:20.207234 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:20.207203 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa321c27-8e97-456c-b99a-a68a87dd5edc" path="/var/lib/kubelet/pods/aa321c27-8e97-456c-b99a-a68a87dd5edc/volumes" Jul 15 19:00:37.397569 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:37.397492 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:38.675018 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.674982 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-22jvw"] Jul 15 19:00:38.675383 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.675290 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa321c27-8e97-456c-b99a-a68a87dd5edc" containerName="manager" Jul 15 19:00:38.675383 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.675301 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa321c27-8e97-456c-b99a-a68a87dd5edc" containerName="manager" Jul 15 19:00:38.675383 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.675360 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa321c27-8e97-456c-b99a-a68a87dd5edc" containerName="manager" Jul 15 19:00:38.678738 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.678718 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" Jul 15 19:00:38.681835 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.681814 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 19:00:38.681941 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.681833 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2qnvb\"" Jul 15 19:00:38.683910 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.683891 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kttmc\" (UniqueName: \"kubernetes.io/projected/0827f76e-ef0a-44c1-87b9-8ee6eef709be-kube-api-access-kttmc\") pod \"payload-processing-f9b8d9b9b-22jvw\" (UID: \"0827f76e-ef0a-44c1-87b9-8ee6eef709be\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" Jul 15 19:00:38.684028 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.683945 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0827f76e-ef0a-44c1-87b9-8ee6eef709be-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-22jvw\" (UID: \"0827f76e-ef0a-44c1-87b9-8ee6eef709be\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" Jul 15 19:00:38.688209 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.688187 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-22jvw"] Jul 15 19:00:38.785141 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.785108 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kttmc\" (UniqueName: \"kubernetes.io/projected/0827f76e-ef0a-44c1-87b9-8ee6eef709be-kube-api-access-kttmc\") pod \"payload-processing-f9b8d9b9b-22jvw\" (UID: \"0827f76e-ef0a-44c1-87b9-8ee6eef709be\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" Jul 15 19:00:38.785319 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.785173 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0827f76e-ef0a-44c1-87b9-8ee6eef709be-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-22jvw\" (UID: \"0827f76e-ef0a-44c1-87b9-8ee6eef709be\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" Jul 15 19:00:38.785829 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.785811 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0827f76e-ef0a-44c1-87b9-8ee6eef709be-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-22jvw\" (UID: \"0827f76e-ef0a-44c1-87b9-8ee6eef709be\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" Jul 15 19:00:38.793081 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.793050 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kttmc\" (UniqueName: \"kubernetes.io/projected/0827f76e-ef0a-44c1-87b9-8ee6eef709be-kube-api-access-kttmc\") pod \"payload-processing-f9b8d9b9b-22jvw\" (UID: \"0827f76e-ef0a-44c1-87b9-8ee6eef709be\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" Jul 15 19:00:38.990827 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:38.990717 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" Jul 15 19:00:39.118424 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:39.118399 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-22jvw"] Jul 15 19:00:39.120578 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:00:39.120548 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0827f76e_ef0a_44c1_87b9_8ee6eef709be.slice/crio-bb6621b76a0d0eac613151be827f132e9b96db3d5072b3fcb45b62277daccd74 WatchSource:0}: Error finding container bb6621b76a0d0eac613151be827f132e9b96db3d5072b3fcb45b62277daccd74: Status 404 returned error can't find the container with id bb6621b76a0d0eac613151be827f132e9b96db3d5072b3fcb45b62277daccd74 Jul 15 19:00:39.557211 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:39.557180 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:39.827392 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:39.827307 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" event={"ID":"0827f76e-ef0a-44c1-87b9-8ee6eef709be","Type":"ContainerStarted","Data":"bb6621b76a0d0eac613151be827f132e9b96db3d5072b3fcb45b62277daccd74"} Jul 15 19:00:41.839311 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:41.839273 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" event={"ID":"0827f76e-ef0a-44c1-87b9-8ee6eef709be","Type":"ContainerStarted","Data":"6258bed440795898d3905ab6950c3aa6e814efe4a0b3c2f3c57d986bae9633a1"} Jul 15 19:00:41.839698 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:41.839400 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" Jul 15 19:00:41.859114 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:41.859069 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" podStartSLOduration=1.8679226789999999 podStartE2EDuration="3.859054276s" totalImagesPullingTime="1.991131597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:00:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:41.856833759 +0000 UTC m=+592.204546080" watchObservedRunningTime="2026-07-15 19:00:41.859054276 +0000 UTC m=+592.206766575" Jul 15 19:00:48.538659 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:48.538605 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:48.635707 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:48.635664 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:50.122039 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:50.122013 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:00:50.122978 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:50.122959 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:00:52.847282 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:52.847251 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-22jvw" Jul 15 19:00:53.667250 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:53.667217 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:53.767803 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:53.767745 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:59.164800 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:59.164761 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:00:59.265049 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:00:59.265016 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:01:03.550269 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:03.550235 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:01:08.228107 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.228079 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26"] Jul 15 19:01:08.231471 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.231452 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.234945 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.234918 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 19:01:08.235081 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.234928 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 19:01:08.235845 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.235826 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fbsnr\"" Jul 15 19:01:08.235952 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.235835 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 15 19:01:08.242930 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.242904 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26"] Jul 15 19:01:08.280600 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.280568 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:01:08.326883 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.326855 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.327004 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.326894 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.327004 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.326917 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86963b61-152e-475c-8b1a-7352c03f6a13-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.327004 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.326981 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.327148 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.327053 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.327148 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.327135 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plp2v\" (UniqueName: \"kubernetes.io/projected/86963b61-152e-475c-8b1a-7352c03f6a13-kube-api-access-plp2v\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.428172 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.428138 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plp2v\" (UniqueName: \"kubernetes.io/projected/86963b61-152e-475c-8b1a-7352c03f6a13-kube-api-access-plp2v\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.428349 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.428184 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.428349 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.428219 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.428349 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.428234 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86963b61-152e-475c-8b1a-7352c03f6a13-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.428349 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.428258 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.428349 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.428309 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.428695 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.428675 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.428760 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.428737 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.428918 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.428891 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.430448 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.430428 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86963b61-152e-475c-8b1a-7352c03f6a13-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.430698 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.430682 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86963b61-152e-475c-8b1a-7352c03f6a13-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.437320 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.437296 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plp2v\" (UniqueName: \"kubernetes.io/projected/86963b61-152e-475c-8b1a-7352c03f6a13-kube-api-access-plp2v\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zkb26\" (UID: \"86963b61-152e-475c-8b1a-7352c03f6a13\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.542827 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.542777 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:08.669100 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.669068 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26"] Jul 15 19:01:08.670193 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:01:08.670166 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86963b61_152e_475c_8b1a_7352c03f6a13.slice/crio-44d80a022307351a04f10bb3e8eaf3ef0b71c2d31c7890bdb5644e7b8e17aff5 WatchSource:0}: Error finding container 44d80a022307351a04f10bb3e8eaf3ef0b71c2d31c7890bdb5644e7b8e17aff5: Status 404 returned error can't find the container with id 44d80a022307351a04f10bb3e8eaf3ef0b71c2d31c7890bdb5644e7b8e17aff5 Jul 15 19:01:08.671490 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.671468 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:01:08.973278 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:08.973192 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" event={"ID":"86963b61-152e-475c-8b1a-7352c03f6a13","Type":"ContainerStarted","Data":"44d80a022307351a04f10bb3e8eaf3ef0b71c2d31c7890bdb5644e7b8e17aff5"} Jul 15 19:01:09.980988 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:09.980950 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" event={"ID":"86963b61-152e-475c-8b1a-7352c03f6a13","Type":"ContainerStarted","Data":"1489ddb61ef8a8591a6b1b5cce6c7c6fd716b6666a30e46265b14afc4959b406"} Jul 15 19:01:10.011882 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:10.011830 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" podStartSLOduration=0.863364375 podStartE2EDuration="2.011811033s" totalImagesPullingTime="1.148446658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:10.011224048 +0000 UTC m=+620.358936370" watchObservedRunningTime="2026-07-15 19:01:10.011811033 +0000 UTC m=+620.359523331" Jul 15 19:01:18.543597 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.543516 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:18.543597 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.543579 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:18.556560 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.556534 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:18.623855 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.623820 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn"] Jul 15 19:01:18.627090 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.627071 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.629254 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.629235 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 15 19:01:18.637643 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.637620 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn"] Jul 15 19:01:18.656048 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.656023 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:01:18.706637 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.706614 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.706754 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.706643 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7958b7e7-964b-4e55-81be-63c7546bb850-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.706754 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.706661 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.706754 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.706679 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.706879 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.706831 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw8zv\" (UniqueName: \"kubernetes.io/projected/7958b7e7-964b-4e55-81be-63c7546bb850-kube-api-access-pw8zv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.706879 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.706864 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.808007 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.807933 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw8zv\" (UniqueName: \"kubernetes.io/projected/7958b7e7-964b-4e55-81be-63c7546bb850-kube-api-access-pw8zv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.808007 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.807970 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.808007 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.808008 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.808273 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.808024 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7958b7e7-964b-4e55-81be-63c7546bb850-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.808273 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.808040 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.808273 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.808079 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.808431 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.808376 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.808512 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.808490 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.808552 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.808527 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.810250 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.810229 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7958b7e7-964b-4e55-81be-63c7546bb850-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.810424 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.810405 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7958b7e7-964b-4e55-81be-63c7546bb850-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.815475 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.815450 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw8zv\" (UniqueName: \"kubernetes.io/projected/7958b7e7-964b-4e55-81be-63c7546bb850-kube-api-access-pw8zv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cp7rn\" (UID: \"7958b7e7-964b-4e55-81be-63c7546bb850\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:18.936642 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:18.936611 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:19.037413 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:19.037388 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zkb26" Jul 15 19:01:19.081004 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:19.080979 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn"] Jul 15 19:01:19.082753 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:01:19.082728 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7958b7e7_964b_4e55_81be_63c7546bb850.slice/crio-d6949ce6c7c1907ae5e1bcca96b2cb3acf9941b6597f3344d4f42fdf5614cd0d WatchSource:0}: Error finding container d6949ce6c7c1907ae5e1bcca96b2cb3acf9941b6597f3344d4f42fdf5614cd0d: Status 404 returned error can't find the container with id d6949ce6c7c1907ae5e1bcca96b2cb3acf9941b6597f3344d4f42fdf5614cd0d Jul 15 19:01:20.032687 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:20.032645 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" event={"ID":"7958b7e7-964b-4e55-81be-63c7546bb850","Type":"ContainerStarted","Data":"d6949ce6c7c1907ae5e1bcca96b2cb3acf9941b6597f3344d4f42fdf5614cd0d"} Jul 15 19:01:21.329619 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:21.329585 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:01:26.523798 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:26.523760 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:01:40.133486 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.133447 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" event={"ID":"7958b7e7-964b-4e55-81be-63c7546bb850","Type":"ContainerStarted","Data":"7c431a2c3c97aefadc413e14988bb64b54ae22586c77d4cea51b3e167792e7a9"} Jul 15 19:01:40.140048 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.140019 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9"] Jul 15 19:01:40.143315 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.143296 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.145612 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.145593 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 15 19:01:40.153305 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.153283 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9"] Jul 15 19:01:40.155794 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.155744 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" podStartSLOduration=1.757090341 podStartE2EDuration="22.155729759s" totalImagesPullingTime="20.398639418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:40.153085365 +0000 UTC m=+650.500797666" watchObservedRunningTime="2026-07-15 19:01:40.155729759 +0000 UTC m=+650.503442057" Jul 15 19:01:40.165976 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.165941 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:01:40.200402 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.200373 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.200509 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.200432 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d57a2481-7549-4855-9a7d-95b5d8bc079d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.200509 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.200487 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.200628 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.200511 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.200628 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.200533 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.200628 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.200603 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwdmc\" (UniqueName: \"kubernetes.io/projected/d57a2481-7549-4855-9a7d-95b5d8bc079d-kube-api-access-jwdmc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.301058 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.301024 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d57a2481-7549-4855-9a7d-95b5d8bc079d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.301259 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.301066 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.301259 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.301086 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.301259 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.301110 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.301259 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.301170 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwdmc\" (UniqueName: \"kubernetes.io/projected/d57a2481-7549-4855-9a7d-95b5d8bc079d-kube-api-access-jwdmc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.301259 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.301242 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.302329 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.302219 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.302329 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.302239 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.302329 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.302299 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.304407 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.304366 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d57a2481-7549-4855-9a7d-95b5d8bc079d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.304513 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.304366 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d57a2481-7549-4855-9a7d-95b5d8bc079d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.311644 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.311609 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwdmc\" (UniqueName: \"kubernetes.io/projected/d57a2481-7549-4855-9a7d-95b5d8bc079d-kube-api-access-jwdmc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9\" (UID: \"d57a2481-7549-4855-9a7d-95b5d8bc079d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.452197 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.452111 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:40.590129 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:40.590104 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9"] Jul 15 19:01:40.591677 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:01:40.591649 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd57a2481_7549_4855_9a7d_95b5d8bc079d.slice/crio-47696690453d7b3383dee00f7baf805682b68e7767f2824f360c6e31c3041c72 WatchSource:0}: Error finding container 47696690453d7b3383dee00f7baf805682b68e7767f2824f360c6e31c3041c72: Status 404 returned error can't find the container with id 47696690453d7b3383dee00f7baf805682b68e7767f2824f360c6e31c3041c72 Jul 15 19:01:41.139663 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:41.139617 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" event={"ID":"d57a2481-7549-4855-9a7d-95b5d8bc079d","Type":"ContainerStarted","Data":"47696690453d7b3383dee00f7baf805682b68e7767f2824f360c6e31c3041c72"} Jul 15 19:01:45.160697 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:45.160660 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" event={"ID":"d57a2481-7549-4855-9a7d-95b5d8bc079d","Type":"ContainerStarted","Data":"031aeaa4211ca35462fed7fa2d7d11d13be38e390aeee7b7cb1b128cd5773753"} Jul 15 19:01:45.179746 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:45.179696 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" podStartSLOduration=1.469331073 podStartE2EDuration="5.179682362s" totalImagesPullingTime="3.710351289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:45.177738897 +0000 UTC m=+655.525451196" watchObservedRunningTime="2026-07-15 19:01:45.179682362 +0000 UTC m=+655.527394661" Jul 15 19:01:48.936908 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:48.936855 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:48.937370 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:48.936933 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:48.954952 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:48.954920 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:49.196005 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:49.195930 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cp7rn" Jul 15 19:01:50.452445 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:50.452415 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:50.452902 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:50.452467 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:50.464836 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:50.464815 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:01:51.205713 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:01:51.205680 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9" Jul 15 19:02:24.625863 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:24.625825 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:02:31.580976 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:31.580942 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7964cb565-h9ffr"] Jul 15 19:02:31.584331 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:31.584314 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7964cb565-h9ffr" Jul 15 19:02:31.594476 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:31.594450 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7964cb565-h9ffr"] Jul 15 19:02:31.634245 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:31.634211 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c5c79352-4fa4-48e2-86f3-a754c08e393d-tls-cert\") pod \"authorino-7964cb565-h9ffr\" (UID: \"c5c79352-4fa4-48e2-86f3-a754c08e393d\") " pod="kuadrant-system/authorino-7964cb565-h9ffr" Jul 15 19:02:31.634406 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:31.634313 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96qtd\" (UniqueName: \"kubernetes.io/projected/c5c79352-4fa4-48e2-86f3-a754c08e393d-kube-api-access-96qtd\") pod \"authorino-7964cb565-h9ffr\" (UID: \"c5c79352-4fa4-48e2-86f3-a754c08e393d\") " pod="kuadrant-system/authorino-7964cb565-h9ffr" Jul 15 19:02:31.735402 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:31.735367 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96qtd\" (UniqueName: \"kubernetes.io/projected/c5c79352-4fa4-48e2-86f3-a754c08e393d-kube-api-access-96qtd\") pod \"authorino-7964cb565-h9ffr\" (UID: \"c5c79352-4fa4-48e2-86f3-a754c08e393d\") " pod="kuadrant-system/authorino-7964cb565-h9ffr" Jul 15 19:02:31.735402 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:31.735414 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c5c79352-4fa4-48e2-86f3-a754c08e393d-tls-cert\") pod \"authorino-7964cb565-h9ffr\" (UID: \"c5c79352-4fa4-48e2-86f3-a754c08e393d\") " pod="kuadrant-system/authorino-7964cb565-h9ffr" Jul 15 19:02:31.737887 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:31.737859 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c5c79352-4fa4-48e2-86f3-a754c08e393d-tls-cert\") pod \"authorino-7964cb565-h9ffr\" (UID: \"c5c79352-4fa4-48e2-86f3-a754c08e393d\") " pod="kuadrant-system/authorino-7964cb565-h9ffr" Jul 15 19:02:31.744055 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:31.744033 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96qtd\" (UniqueName: \"kubernetes.io/projected/c5c79352-4fa4-48e2-86f3-a754c08e393d-kube-api-access-96qtd\") pod \"authorino-7964cb565-h9ffr\" (UID: \"c5c79352-4fa4-48e2-86f3-a754c08e393d\") " pod="kuadrant-system/authorino-7964cb565-h9ffr" Jul 15 19:02:31.893931 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:31.893842 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7964cb565-h9ffr" Jul 15 19:02:32.032044 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:32.032021 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7964cb565-h9ffr"] Jul 15 19:02:32.034035 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:02:32.034010 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5c79352_4fa4_48e2_86f3_a754c08e393d.slice/crio-b161f4e13a83f8465d720a69ed0f38ea5f4ed680e9d6dca15eb9da11a265d202 WatchSource:0}: Error finding container b161f4e13a83f8465d720a69ed0f38ea5f4ed680e9d6dca15eb9da11a265d202: Status 404 returned error can't find the container with id b161f4e13a83f8465d720a69ed0f38ea5f4ed680e9d6dca15eb9da11a265d202 Jul 15 19:02:32.393468 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:32.393444 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7964cb565-h9ffr" event={"ID":"c5c79352-4fa4-48e2-86f3-a754c08e393d","Type":"ContainerStarted","Data":"b161f4e13a83f8465d720a69ed0f38ea5f4ed680e9d6dca15eb9da11a265d202"} Jul 15 19:02:33.401420 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:33.401387 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7964cb565-h9ffr" event={"ID":"c5c79352-4fa4-48e2-86f3-a754c08e393d","Type":"ContainerStarted","Data":"45fa72d66aa02a2022f7ae678550aee9f0284310f4c16bf2faf7d51efde603f6"} Jul 15 19:02:33.416687 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:33.416631 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7964cb565-h9ffr" podStartSLOduration=2.087480384 podStartE2EDuration="2.416617213s" totalImagesPullingTime="329.136829ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:02:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:02:33.415993302 +0000 UTC m=+703.763705625" watchObservedRunningTime="2026-07-15 19:02:33.416617213 +0000 UTC m=+703.764329512" Jul 15 19:02:33.446465 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:33.446428 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d4c5f7b88-tndsb"] Jul 15 19:02:33.446646 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:33.446607 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" podUID="4321c481-c77f-42c3-b396-e260a61c20ff" containerName="authorino" containerID="cri-o://ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1" gracePeriod=30 Jul 15 19:02:33.696272 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:33.696249 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" Jul 15 19:02:33.755656 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:33.755617 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4321c481-c77f-42c3-b396-e260a61c20ff-tls-cert\") pod \"4321c481-c77f-42c3-b396-e260a61c20ff\" (UID: \"4321c481-c77f-42c3-b396-e260a61c20ff\") " Jul 15 19:02:33.755866 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:33.755663 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9snf\" (UniqueName: \"kubernetes.io/projected/4321c481-c77f-42c3-b396-e260a61c20ff-kube-api-access-m9snf\") pod \"4321c481-c77f-42c3-b396-e260a61c20ff\" (UID: \"4321c481-c77f-42c3-b396-e260a61c20ff\") " Jul 15 19:02:33.757771 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:33.757726 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4321c481-c77f-42c3-b396-e260a61c20ff-kube-api-access-m9snf" (OuterVolumeSpecName: "kube-api-access-m9snf") pod "4321c481-c77f-42c3-b396-e260a61c20ff" (UID: "4321c481-c77f-42c3-b396-e260a61c20ff"). InnerVolumeSpecName "kube-api-access-m9snf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:02:33.766069 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:33.766037 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4321c481-c77f-42c3-b396-e260a61c20ff-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4321c481-c77f-42c3-b396-e260a61c20ff" (UID: "4321c481-c77f-42c3-b396-e260a61c20ff"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:02:33.856473 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:33.856426 2574 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4321c481-c77f-42c3-b396-e260a61c20ff-tls-cert\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:02:33.856473 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:33.856472 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9snf\" (UniqueName: \"kubernetes.io/projected/4321c481-c77f-42c3-b396-e260a61c20ff-kube-api-access-m9snf\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:02:34.408916 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:34.408875 2574 generic.go:356] "Generic (PLEG): container finished" podID="4321c481-c77f-42c3-b396-e260a61c20ff" containerID="ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1" exitCode=0 Jul 15 19:02:34.409383 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:34.408933 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" Jul 15 19:02:34.409383 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:34.408971 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" event={"ID":"4321c481-c77f-42c3-b396-e260a61c20ff","Type":"ContainerDied","Data":"ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1"} Jul 15 19:02:34.409383 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:34.409018 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d4c5f7b88-tndsb" event={"ID":"4321c481-c77f-42c3-b396-e260a61c20ff","Type":"ContainerDied","Data":"1e225a3ebae308228e5521bdf91875f8b0f6fdc168eb4fb441ab36c867bc098f"} Jul 15 19:02:34.409383 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:34.409052 2574 scope.go:117] "RemoveContainer" containerID="ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1" Jul 15 19:02:34.419376 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:34.419355 2574 scope.go:117] "RemoveContainer" containerID="ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1" Jul 15 19:02:34.419678 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:02:34.419658 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1\": container with ID starting with ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1 not found: ID does not exist" containerID="ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1" Jul 15 19:02:34.419747 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:34.419691 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1"} err="failed to get container status \"ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1\": rpc error: code = NotFound desc = could not find container \"ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1\": container with ID starting with ea4ffb3b15955da4f92cdbd6cc2ab2b1489e66a72d76a2a5a2f38109dc4077c1 not found: ID does not exist" Jul 15 19:02:34.430222 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:34.430191 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d4c5f7b88-tndsb"] Jul 15 19:02:34.443154 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:34.443124 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d4c5f7b88-tndsb"] Jul 15 19:02:34.468833 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:34.468717 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:02:34.588976 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:34.588932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:02:36.206177 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:36.206144 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4321c481-c77f-42c3-b396-e260a61c20ff" path="/var/lib/kubelet/pods/4321c481-c77f-42c3-b396-e260a61c20ff/volumes" Jul 15 19:02:36.654362 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:36.654329 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:02:36.767664 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:02:36.767630 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:03:50.952468 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:50.952433 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:03:51.107030 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.106991 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dcfcc7558-hwx45"] Jul 15 19:03:51.107312 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.107243 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" podUID="d48a04c9-9e35-49b4-a9b5-694d97dc1043" containerName="manager" containerID="cri-o://1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546" gracePeriod=30 Jul 15 19:03:51.354705 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.354682 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:03:51.412693 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.412656 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d48a04c9-9e35-49b4-a9b5-694d97dc1043-webhook-cert\") pod \"d48a04c9-9e35-49b4-a9b5-694d97dc1043\" (UID: \"d48a04c9-9e35-49b4-a9b5-694d97dc1043\") " Jul 15 19:03:51.412864 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.412736 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq5kd\" (UniqueName: \"kubernetes.io/projected/d48a04c9-9e35-49b4-a9b5-694d97dc1043-kube-api-access-vq5kd\") pod \"d48a04c9-9e35-49b4-a9b5-694d97dc1043\" (UID: \"d48a04c9-9e35-49b4-a9b5-694d97dc1043\") " Jul 15 19:03:51.414666 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.414638 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d48a04c9-9e35-49b4-a9b5-694d97dc1043-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d48a04c9-9e35-49b4-a9b5-694d97dc1043" (UID: "d48a04c9-9e35-49b4-a9b5-694d97dc1043"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:03:51.414950 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.414928 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d48a04c9-9e35-49b4-a9b5-694d97dc1043-kube-api-access-vq5kd" (OuterVolumeSpecName: "kube-api-access-vq5kd") pod "d48a04c9-9e35-49b4-a9b5-694d97dc1043" (UID: "d48a04c9-9e35-49b4-a9b5-694d97dc1043"). InnerVolumeSpecName "kube-api-access-vq5kd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:03:51.513597 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.513575 2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d48a04c9-9e35-49b4-a9b5-694d97dc1043-webhook-cert\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:03:51.513597 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.513599 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq5kd\" (UniqueName: \"kubernetes.io/projected/d48a04c9-9e35-49b4-a9b5-694d97dc1043-kube-api-access-vq5kd\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:03:51.812089 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.812000 2574 generic.go:356] "Generic (PLEG): container finished" podID="d48a04c9-9e35-49b4-a9b5-694d97dc1043" containerID="1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546" exitCode=0 Jul 15 19:03:51.812089 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.812043 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" Jul 15 19:03:51.812268 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.812091 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" event={"ID":"d48a04c9-9e35-49b4-a9b5-694d97dc1043","Type":"ContainerDied","Data":"1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546"} Jul 15 19:03:51.812268 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.812148 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dcfcc7558-hwx45" event={"ID":"d48a04c9-9e35-49b4-a9b5-694d97dc1043","Type":"ContainerDied","Data":"554f471cc7fd31ff2c95b1bf0f4bb023f5a85328e995f0b5023d6ec0022e0654"} Jul 15 19:03:51.812268 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.812175 2574 scope.go:117] "RemoveContainer" containerID="1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546" Jul 15 19:03:51.821264 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.821246 2574 scope.go:117] "RemoveContainer" containerID="1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546" Jul 15 19:03:51.821506 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:03:51.821488 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546\": container with ID starting with 1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546 not found: ID does not exist" containerID="1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546" Jul 15 19:03:51.821552 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.821514 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546"} err="failed to get container status \"1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546\": rpc error: code = NotFound desc = could not find container \"1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546\": container with ID starting with 1e362fd18cfbd5e204565111d7c95769e977e69277416dd760c28c979d18d546 not found: ID does not exist" Jul 15 19:03:51.836066 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.836042 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dcfcc7558-hwx45"] Jul 15 19:03:51.840261 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:51.840242 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7dcfcc7558-hwx45"] Jul 15 19:03:52.206275 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.206183 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d48a04c9-9e35-49b4-a9b5-694d97dc1043" path="/var/lib/kubelet/pods/d48a04c9-9e35-49b4-a9b5-694d97dc1043/volumes" Jul 15 19:03:52.554970 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.554936 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:03:52.674262 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.674231 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7dcfcc7558-xnh2k"] Jul 15 19:03:52.674566 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.674553 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4321c481-c77f-42c3-b396-e260a61c20ff" containerName="authorino" Jul 15 19:03:52.674612 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.674568 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4321c481-c77f-42c3-b396-e260a61c20ff" containerName="authorino" Jul 15 19:03:52.674612 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.674584 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d48a04c9-9e35-49b4-a9b5-694d97dc1043" containerName="manager" Jul 15 19:03:52.674612 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.674589 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d48a04c9-9e35-49b4-a9b5-694d97dc1043" containerName="manager" Jul 15 19:03:52.674705 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.674641 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d48a04c9-9e35-49b4-a9b5-694d97dc1043" containerName="manager" Jul 15 19:03:52.674705 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.674651 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4321c481-c77f-42c3-b396-e260a61c20ff" containerName="authorino" Jul 15 19:03:52.678920 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.678904 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" Jul 15 19:03:52.682050 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.682024 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qqrkm\"" Jul 15 19:03:52.682167 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.682023 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 19:03:52.682167 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.682032 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 19:03:52.686410 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.686177 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dcfcc7558-xnh2k"] Jul 15 19:03:52.824913 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.824822 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7x9b\" (UniqueName: \"kubernetes.io/projected/180331a3-7b84-4974-8c02-e91cc5148a9b-kube-api-access-g7x9b\") pod \"maas-controller-7dcfcc7558-xnh2k\" (UID: \"180331a3-7b84-4974-8c02-e91cc5148a9b\") " pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" Jul 15 19:03:52.824913 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.824865 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/180331a3-7b84-4974-8c02-e91cc5148a9b-webhook-cert\") pod \"maas-controller-7dcfcc7558-xnh2k\" (UID: \"180331a3-7b84-4974-8c02-e91cc5148a9b\") " pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" Jul 15 19:03:52.926106 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.926076 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/180331a3-7b84-4974-8c02-e91cc5148a9b-webhook-cert\") pod \"maas-controller-7dcfcc7558-xnh2k\" (UID: \"180331a3-7b84-4974-8c02-e91cc5148a9b\") " pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" Jul 15 19:03:52.926259 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.926179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7x9b\" (UniqueName: \"kubernetes.io/projected/180331a3-7b84-4974-8c02-e91cc5148a9b-kube-api-access-g7x9b\") pod \"maas-controller-7dcfcc7558-xnh2k\" (UID: \"180331a3-7b84-4974-8c02-e91cc5148a9b\") " pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" Jul 15 19:03:52.928523 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.928492 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/180331a3-7b84-4974-8c02-e91cc5148a9b-webhook-cert\") pod \"maas-controller-7dcfcc7558-xnh2k\" (UID: \"180331a3-7b84-4974-8c02-e91cc5148a9b\") " pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" Jul 15 19:03:52.934269 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.934241 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7x9b\" (UniqueName: \"kubernetes.io/projected/180331a3-7b84-4974-8c02-e91cc5148a9b-kube-api-access-g7x9b\") pod \"maas-controller-7dcfcc7558-xnh2k\" (UID: \"180331a3-7b84-4974-8c02-e91cc5148a9b\") " pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" Jul 15 19:03:52.989606 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:52.989581 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" Jul 15 19:03:53.118693 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:53.118665 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dcfcc7558-xnh2k"] Jul 15 19:03:53.120246 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:03:53.120217 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180331a3_7b84_4974_8c02_e91cc5148a9b.slice/crio-44608ab5722fbc89a37345f5f8705392ac7751190764884faa1990d78c273264 WatchSource:0}: Error finding container 44608ab5722fbc89a37345f5f8705392ac7751190764884faa1990d78c273264: Status 404 returned error can't find the container with id 44608ab5722fbc89a37345f5f8705392ac7751190764884faa1990d78c273264 Jul 15 19:03:53.824871 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:53.824838 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" event={"ID":"180331a3-7b84-4974-8c02-e91cc5148a9b","Type":"ContainerStarted","Data":"30158dc46968d7c56c630759c169b0a5dd0e7ef302abd88abc673242c1bd76fc"} Jul 15 19:03:53.825360 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:53.824895 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" event={"ID":"180331a3-7b84-4974-8c02-e91cc5148a9b","Type":"ContainerStarted","Data":"44608ab5722fbc89a37345f5f8705392ac7751190764884faa1990d78c273264"} Jul 15 19:03:53.825360 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:53.824940 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" Jul 15 19:03:53.842536 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:53.842494 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" podStartSLOduration=1.348949864 podStartE2EDuration="1.842480048s" totalImagesPullingTime="493.530184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:03:53.840609715 +0000 UTC m=+784.188322014" watchObservedRunningTime="2026-07-15 19:03:53.842480048 +0000 UTC m=+784.190192347" Jul 15 19:03:59.924994 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:03:59.924960 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:04:04.832618 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:04:04.832584 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7dcfcc7558-xnh2k" Jul 15 19:04:31.641814 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:04:31.641765 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:04:31.730486 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:04:31.730449 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:04:31.885083 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:04:31.885047 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:04:36.354841 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:04:36.354804 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:04:38.114220 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:04:38.114181 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:04:38.230002 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:04:38.229963 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:04:38.824489 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:04:38.824458 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:04:38.925776 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:04:38.925739 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:04:44.673507 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:04:44.673468 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:04:44.770998 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:04:44.770965 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:05:50.162591 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:05:50.162561 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:05:50.164082 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:05:50.164057 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:05:52.228474 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:05:52.228428 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:06:39.656180 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:06:39.656126 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:10:13.638726 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:10:13.638632 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:10:13.716080 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:10:13.716050 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:10:50.196533 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:10:50.196499 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:10:50.198509 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:10:50.198491 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:11:36.755441 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:11:36.755407 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:12:58.169544 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:12:58.169505 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:13:55.510537 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:13:55.510499 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:13:55.562553 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:13:55.562519 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:15:49.600157 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:15:49.600124 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:15:50.235798 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:15:50.235750 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:15:50.238215 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:15:50.238197 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:15:56.483182 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:15:56.483148 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:04.753690 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:04.753657 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:09.889117 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:09.889077 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:13.002345 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:13.002273 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:13.101070 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:13.101036 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:13.444039 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:13.444001 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:19.352674 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:19.352625 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:19.438210 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:19.438182 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:19.640580 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:19.640472 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:19.794122 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:19.794088 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:20.950830 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:20.950802 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:21.049544 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:21.049504 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:21.146022 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:21.145984 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:21.468972 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:21.468941 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:21.641970 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:21.641936 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:21.752938 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:21.752914 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:16:23.437871 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:16:23.437832 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:36.420768 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:36.420738 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:41.010585 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:41.010549 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:46.734905 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:46.734831 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:46.837829 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:46.837775 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:51.337061 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:51.337026 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:51.613117 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:51.613034 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:51.723121 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:51.723088 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:51.911982 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:51.911896 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:53.185285 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:53.185250 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:53.463511 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:53.463427 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:53.851880 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:53.851847 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:54.136939 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:54.136853 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:54.253183 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:54.253150 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:58.024904 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:58.024856 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:17:58.110899 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:17:58.110863 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:18:01.491598 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:18:01.491573 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:18:57.781906 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:18:57.781866 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:18:57.901586 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:18:57.901552 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:03.400678 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:03.400593 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:03.971403 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:03.971370 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:04.119635 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:04.119599 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:09.538265 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:09.538233 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:09.657357 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:09.657325 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:09.826485 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:09.826402 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:09.918465 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:09.918434 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:11.002666 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:11.002630 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:11.106167 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:11.106140 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:11.216770 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:11.216740 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:11.517978 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:11.517951 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:11.918349 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:11.918270 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:12.019938 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:12.019900 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:13.957665 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:13.957613 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:14.170468 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:14.170368 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:14.313345 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:14.313310 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:17.108020 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:17.107989 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:19:17.277702 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:19:17.277670 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:26.821461 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:26.821424 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:31.258928 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:31.258889 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:37.177922 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:37.177890 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:37.280595 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:37.280565 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:42.289997 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:42.289956 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:42.606651 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:42.606613 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:43.766645 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:43.766598 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:43.932989 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:43.932944 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:44.052267 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:44.052174 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:44.201338 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:44.201306 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:44.629310 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:44.629278 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:44.737323 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:44.737276 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:50.270580 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:50.270552 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:20:50.274602 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:50.274581 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:20:53.951682 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:53.951653 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:54.053589 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:54.053558 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:20:57.635459 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:20:57.635423 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:21:38.092147 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:21:38.092109 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:21:48.993847 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:21:48.993813 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:21:53.357915 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:21:53.357883 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:21:59.437573 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:21:59.437541 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:21:59.537573 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:21:59.537539 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:03.573801 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:03.573759 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:04.179183 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:04.179144 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:04.275443 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:04.275415 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:04.376648 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:04.376612 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:06.006835 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:06.006804 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:10.198738 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:10.198699 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:13.544138 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:13.544063 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:13.680240 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:13.680201 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:13.787030 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:13.787000 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:14.292931 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:14.292900 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:37.441187 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:37.441156 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:37.565906 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:37.565864 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:40.325298 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:40.325257 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:42.192848 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:42.192813 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:46.700335 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:46.700304 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:46.783267 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:46.783231 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:22:46.898667 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:22:46.898638 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:02.909494 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:02.909460 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:08.921034 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:08.920996 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:09.107378 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:09.107345 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:09.200460 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:09.200381 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:19.700759 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:19.700712 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:20.528499 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:20.528464 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:26.945089 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:26.945057 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:27.053066 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:27.053031 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:27.177326 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:27.177295 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:27.368705 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:27.368672 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:28.078594 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:28.078562 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:28.303027 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:28.302995 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:28.409068 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:28.408991 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:29.963536 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:29.963505 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:36.178102 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:36.178073 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:36.277407 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:36.277375 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:36.913121 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:36.913090 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:43.341834 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:43.341728 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:43.442364 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:43.442328 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:43.602165 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:43.602083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:43.945250 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:43.945156 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:44.452578 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:44.452545 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:47.492194 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:47.492163 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:47.588070 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:47.588040 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:47.774363 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:47.774328 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:47.863290 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:47.863260 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:48.378089 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:48.378058 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:48.464295 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:48.464262 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:51.892657 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:51.892622 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:52.667149 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:52.667118 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:52.763523 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:52.763495 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:53.121683 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:53.121650 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:59.361547 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:59.361514 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:23:59.495489 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:23:59.495453 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:10.356189 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:10.356157 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:14.986778 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:14.986742 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:15.345454 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:15.345419 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:15.448573 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:15.448523 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:21.081563 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:21.081523 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:21.145865 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:21.145832 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:25.664853 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:25.664821 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:25.884511 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:25.884483 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:26.011110 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:26.011079 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:26.112369 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:26.112334 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:26.296935 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:26.296836 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:32.523342 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:32.523310 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:32.769485 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:32.769446 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:32.903047 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:32.902964 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:32.982327 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:32.982291 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:38.873825 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:38.873768 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:38.938318 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:38.938254 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:45.779668 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:45.779580 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:46.234488 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:46.234406 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:46.507172 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:46.507131 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:48.733404 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:48.733349 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:48.822677 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:48.822643 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:48.905466 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:48.905428 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:49.018986 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:49.018951 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:49.694484 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:49.694450 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:49.786725 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:49.786693 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:24:53.256214 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:24:53.256168 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:25:00.930559 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:00.930519 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l"] Jul 15 19:25:00.933994 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:00.933971 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:00.937121 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:00.937093 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-52712cae\"/\"kube-root-ca.crt\"" Jul 15 19:25:00.937288 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:00.937162 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-52712cae\"/\"openshift-service-ca.crt\"" Jul 15 19:25:00.937288 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:00.937162 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-52712cae\"/\"e2e-lifecycle-model-52712cae-kserve-self-signed-certs\"" Jul 15 19:25:00.937288 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:00.937229 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-52712cae\"/\"default-dockercfg-fzvcx\"" Jul 15 19:25:00.944063 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:00.944036 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l"] Jul 15 19:25:01.070647 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.070599 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-model-cache\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.070647 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.070649 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-home\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.070910 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.070726 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-dshm\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.070910 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.070748 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc5f3892-f6cd-4367-9065-0530a36d5d69-tls-certs\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.070910 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.070765 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqm8d\" (UniqueName: \"kubernetes.io/projected/dc5f3892-f6cd-4367-9065-0530a36d5d69-kube-api-access-sqm8d\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.070910 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.070887 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-tmp-dir\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.172303 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.172261 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-model-cache\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.172303 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.172303 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-home\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.172525 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.172336 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-dshm\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.172525 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.172352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc5f3892-f6cd-4367-9065-0530a36d5d69-tls-certs\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.172525 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.172369 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqm8d\" (UniqueName: \"kubernetes.io/projected/dc5f3892-f6cd-4367-9065-0530a36d5d69-kube-api-access-sqm8d\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.172525 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.172404 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-tmp-dir\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.172715 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.172656 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-model-cache\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.172807 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.172770 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-home\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.172904 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.172884 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-tmp-dir\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.174645 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.174617 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-dshm\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.175032 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.175014 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc5f3892-f6cd-4367-9065-0530a36d5d69-tls-certs\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.183815 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.183730 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqm8d\" (UniqueName: \"kubernetes.io/projected/dc5f3892-f6cd-4367-9065-0530a36d5d69-kube-api-access-sqm8d\") pod \"e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.244216 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.244180 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:01.379698 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.379675 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l"] Jul 15 19:25:01.381055 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:25:01.381027 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc5f3892_f6cd_4367_9065_0530a36d5d69.slice/crio-109e138ade3f603463f2e639ced2fd2a8109db15bee0e6dfb06f64deb55882d9 WatchSource:0}: Error finding container 109e138ade3f603463f2e639ced2fd2a8109db15bee0e6dfb06f64deb55882d9: Status 404 returned error can't find the container with id 109e138ade3f603463f2e639ced2fd2a8109db15bee0e6dfb06f64deb55882d9 Jul 15 19:25:01.448972 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:01.448949 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:25:02.170019 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:02.169980 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" event={"ID":"dc5f3892-f6cd-4367-9065-0530a36d5d69","Type":"ContainerStarted","Data":"04aebdfa0f56121e00af283202d9253238760945b8ead1ee9df81c8ae2b8e285"} Jul 15 19:25:02.170401 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:02.170027 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" event={"ID":"dc5f3892-f6cd-4367-9065-0530a36d5d69","Type":"ContainerStarted","Data":"109e138ade3f603463f2e639ced2fd2a8109db15bee0e6dfb06f64deb55882d9"} Jul 15 19:25:02.187358 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:02.187314 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" podStartSLOduration=2.187297594 podStartE2EDuration="2.187297594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:02.185959174 +0000 UTC m=+2052.533671473" watchObservedRunningTime="2026-07-15 19:25:02.187297594 +0000 UTC m=+2052.535009893" Jul 15 19:25:11.245130 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:11.245096 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:11.245130 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:11.245138 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:11.257762 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:11.257732 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:12.098121 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:12.098086 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:25:12.231551 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:12.231522 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:25:50.305702 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:50.305670 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:25:50.311929 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:50.311909 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:25:58.017025 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:25:58.016991 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:26:02.288522 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:02.288481 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:26:07.314199 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:07.314165 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l"] Jul 15 19:26:07.498988 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:07.498952 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:26:22.334102 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.334034 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" podUID="dc5f3892-f6cd-4367-9065-0530a36d5d69" containerName="main" containerID="cri-o://04aebdfa0f56121e00af283202d9253238760945b8ead1ee9df81c8ae2b8e285" gracePeriod=45 Jul 15 19:26:22.368068 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:26:22.368038 2574 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc5f3892_f6cd_4367_9065_0530a36d5d69.slice/crio-04aebdfa0f56121e00af283202d9253238760945b8ead1ee9df81c8ae2b8e285.scope\": RecentStats: unable to find data in memory cache]" Jul 15 19:26:22.569411 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.569379 2574 generic.go:356] "Generic (PLEG): container finished" podID="dc5f3892-f6cd-4367-9065-0530a36d5d69" containerID="04aebdfa0f56121e00af283202d9253238760945b8ead1ee9df81c8ae2b8e285" exitCode=0 Jul 15 19:26:22.569598 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.569429 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" event={"ID":"dc5f3892-f6cd-4367-9065-0530a36d5d69","Type":"ContainerDied","Data":"04aebdfa0f56121e00af283202d9253238760945b8ead1ee9df81c8ae2b8e285"} Jul 15 19:26:22.590779 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.590712 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:26:22.683755 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.683719 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-home\") pod \"dc5f3892-f6cd-4367-9065-0530a36d5d69\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " Jul 15 19:26:22.683934 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.683806 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-tmp-dir\") pod \"dc5f3892-f6cd-4367-9065-0530a36d5d69\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " Jul 15 19:26:22.683934 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.683841 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqm8d\" (UniqueName: \"kubernetes.io/projected/dc5f3892-f6cd-4367-9065-0530a36d5d69-kube-api-access-sqm8d\") pod \"dc5f3892-f6cd-4367-9065-0530a36d5d69\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " Jul 15 19:26:22.683934 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.683878 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-dshm\") pod \"dc5f3892-f6cd-4367-9065-0530a36d5d69\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " Jul 15 19:26:22.683934 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.683904 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc5f3892-f6cd-4367-9065-0530a36d5d69-tls-certs\") pod \"dc5f3892-f6cd-4367-9065-0530a36d5d69\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " Jul 15 19:26:22.684166 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.683950 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-model-cache\") pod \"dc5f3892-f6cd-4367-9065-0530a36d5d69\" (UID: \"dc5f3892-f6cd-4367-9065-0530a36d5d69\") " Jul 15 19:26:22.684166 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.683903 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-home" (OuterVolumeSpecName: "home") pod "dc5f3892-f6cd-4367-9065-0530a36d5d69" (UID: "dc5f3892-f6cd-4367-9065-0530a36d5d69"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:26:22.684166 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.684101 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dc5f3892-f6cd-4367-9065-0530a36d5d69" (UID: "dc5f3892-f6cd-4367-9065-0530a36d5d69"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:26:22.684316 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.684211 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-model-cache" (OuterVolumeSpecName: "model-cache") pod "dc5f3892-f6cd-4367-9065-0530a36d5d69" (UID: "dc5f3892-f6cd-4367-9065-0530a36d5d69"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:26:22.684316 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.684218 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-home\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:26:22.684316 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.684267 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-tmp-dir\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:26:22.686103 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.686073 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc5f3892-f6cd-4367-9065-0530a36d5d69-kube-api-access-sqm8d" (OuterVolumeSpecName: "kube-api-access-sqm8d") pod "dc5f3892-f6cd-4367-9065-0530a36d5d69" (UID: "dc5f3892-f6cd-4367-9065-0530a36d5d69"). InnerVolumeSpecName "kube-api-access-sqm8d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:26:22.686208 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.686105 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc5f3892-f6cd-4367-9065-0530a36d5d69-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dc5f3892-f6cd-4367-9065-0530a36d5d69" (UID: "dc5f3892-f6cd-4367-9065-0530a36d5d69"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:26:22.686208 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.686160 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-dshm" (OuterVolumeSpecName: "dshm") pod "dc5f3892-f6cd-4367-9065-0530a36d5d69" (UID: "dc5f3892-f6cd-4367-9065-0530a36d5d69"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:26:22.784650 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.784613 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqm8d\" (UniqueName: \"kubernetes.io/projected/dc5f3892-f6cd-4367-9065-0530a36d5d69-kube-api-access-sqm8d\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:26:22.784650 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.784642 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-dshm\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:26:22.784650 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.784655 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc5f3892-f6cd-4367-9065-0530a36d5d69-tls-certs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:26:22.784987 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:22.784666 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc5f3892-f6cd-4367-9065-0530a36d5d69-model-cache\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:26:23.576221 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:23.576186 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" event={"ID":"dc5f3892-f6cd-4367-9065-0530a36d5d69","Type":"ContainerDied","Data":"109e138ade3f603463f2e639ced2fd2a8109db15bee0e6dfb06f64deb55882d9"} Jul 15 19:26:23.576662 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:23.576236 2574 scope.go:117] "RemoveContainer" containerID="04aebdfa0f56121e00af283202d9253238760945b8ead1ee9df81c8ae2b8e285" Jul 15 19:26:23.576662 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:23.576240 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l" Jul 15 19:26:23.611100 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:23.611075 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l"] Jul 15 19:26:23.614282 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:23.614253 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-52712cae/e2e-lifecycle-model-52712cae-kserve-5694dd7f4c-xss5l"] Jul 15 19:26:24.206568 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:24.206537 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc5f3892-f6cd-4367-9065-0530a36d5d69" path="/var/lib/kubelet/pods/dc5f3892-f6cd-4367-9065-0530a36d5d69/volumes" Jul 15 19:26:44.614168 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:44.614091 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:26:46.693384 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:46.693349 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:26:52.726057 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:52.726018 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:26:52.826396 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:26:52.826354 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:04.677052 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:04.677018 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:04.912929 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:04.912890 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:06.002205 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:06.002169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:06.098035 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:06.098008 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:06.340569 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:06.340538 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:06.974884 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:06.974850 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:07.375046 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:07.375005 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:07.476334 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:07.476295 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:08.891849 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:08.891817 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:14.815302 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:14.815262 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:14.913282 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:14.913238 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:15.496567 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:15.496537 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:21.453526 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:21.453496 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:21.563542 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:21.563509 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:21.680546 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:21.680512 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:32.201876 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:32.201842 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:32.571335 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:32.571284 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:53.536966 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:53.536929 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:59.393093 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:59.393045 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:59.465627 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:59.465597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:27:59.756070 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:27:59.756040 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:00.089605 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:00.089524 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:00.339198 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:00.339161 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:02.743284 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:02.743251 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:02.837209 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:02.837156 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:02.970989 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:02.970941 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:03.067922 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:03.067880 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:03.466066 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:03.465970 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:03.613184 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:03.613153 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:07.706861 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:07.706826 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:07.815328 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:07.815255 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:07.937013 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:07.936981 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:08.162881 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:08.162849 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:10.798817 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:10.798760 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:10.884687 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:10.884627 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:11.059016 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:11.058926 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:11.157121 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:11.157090 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:11.585313 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:11.585248 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:11.691615 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:11.691589 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:13.027570 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:13.027487 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:15.299321 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:15.299291 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:24.431619 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.431557 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:24.455616 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.455584 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7"] Jul 15 19:28:24.455954 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.455941 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc5f3892-f6cd-4367-9065-0530a36d5d69" containerName="main" Jul 15 19:28:24.456011 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.455955 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc5f3892-f6cd-4367-9065-0530a36d5d69" containerName="main" Jul 15 19:28:24.456047 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.456025 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc5f3892-f6cd-4367-9065-0530a36d5d69" containerName="main" Jul 15 19:28:24.459243 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.459222 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.461765 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.461747 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7b6049\"/\"test-model-7b6049-kserve-self-signed-certs\"" Jul 15 19:28:24.462686 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.462668 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7b6049\"/\"openshift-service-ca.crt\"" Jul 15 19:28:24.462812 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.462750 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7b6049\"/\"kube-root-ca.crt\"" Jul 15 19:28:24.462903 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.462882 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7b6049\"/\"default-dockercfg-rnbtg\"" Jul 15 19:28:24.468763 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.468743 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7"] Jul 15 19:28:24.509840 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.509815 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7rms\" (UniqueName: \"kubernetes.io/projected/525298c4-d14e-4393-ae35-ad35d10852e5-kube-api-access-h7rms\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.510072 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.509851 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-dshm\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.510072 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.509885 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-tmp-dir\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.510072 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.510039 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/525298c4-d14e-4393-ae35-ad35d10852e5-tls-certs\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.510209 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.510091 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-home\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.510209 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.510115 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-model-cache\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.527942 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.527911 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:24.610551 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.610520 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7rms\" (UniqueName: \"kubernetes.io/projected/525298c4-d14e-4393-ae35-ad35d10852e5-kube-api-access-h7rms\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.610551 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.610553 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-dshm\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.610719 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.610585 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-tmp-dir\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.610719 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.610654 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/525298c4-d14e-4393-ae35-ad35d10852e5-tls-certs\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.610719 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.610684 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-home\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.610719 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.610706 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-model-cache\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.611066 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.611045 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-tmp-dir\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.611136 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.611067 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-home\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.611136 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.611109 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-model-cache\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.612820 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.612798 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-dshm\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.613079 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.613063 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/525298c4-d14e-4393-ae35-ad35d10852e5-tls-certs\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.618348 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.618325 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7rms\" (UniqueName: \"kubernetes.io/projected/525298c4-d14e-4393-ae35-ad35d10852e5-kube-api-access-h7rms\") pod \"test-model-7b6049-kserve-56c78556bf-q9cv7\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.627222 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.627190 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:24.771024 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.770987 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:24.903474 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:24.903449 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7"] Jul 15 19:28:24.905197 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:28:24.905162 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod525298c4_d14e_4393_ae35_ad35d10852e5.slice/crio-44a784ce6f3290c4079df0eafef0661142f84e7b6805ef06bc3fabadc1cb98d5 WatchSource:0}: Error finding container 44a784ce6f3290c4079df0eafef0661142f84e7b6805ef06bc3fabadc1cb98d5: Status 404 returned error can't find the container with id 44a784ce6f3290c4079df0eafef0661142f84e7b6805ef06bc3fabadc1cb98d5 Jul 15 19:28:25.181624 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:25.181535 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" event={"ID":"525298c4-d14e-4393-ae35-ad35d10852e5","Type":"ContainerStarted","Data":"d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001"} Jul 15 19:28:25.181624 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:25.181573 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" event={"ID":"525298c4-d14e-4393-ae35-ad35d10852e5","Type":"ContainerStarted","Data":"44a784ce6f3290c4079df0eafef0661142f84e7b6805ef06bc3fabadc1cb98d5"} Jul 15 19:28:25.198699 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:25.198648 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" podStartSLOduration=1.198635794 podStartE2EDuration="1.198635794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:28:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:25.19741097 +0000 UTC m=+2255.545123288" watchObservedRunningTime="2026-07-15 19:28:25.198635794 +0000 UTC m=+2255.546348093" Jul 15 19:28:32.355737 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:32.355701 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:32.453484 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:32.453449 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:34.772163 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:34.772128 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:34.772695 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:34.772187 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:34.784717 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:34.784687 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:35.247348 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:35.247265 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:28:43.527058 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.527023 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn"] Jul 15 19:28:43.530535 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.530517 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.533730 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.533706 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1dfe1f\"/\"kube-root-ca.crt\"" Jul 15 19:28:43.533730 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.533717 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1dfe1f\"/\"openshift-service-ca.crt\"" Jul 15 19:28:43.533936 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.533711 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1dfe1f\"/\"default-dockercfg-fcrzc\"" Jul 15 19:28:43.533936 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.533717 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1dfe1f\"/\"test-model-1dfe1f-kserve-self-signed-certs\"" Jul 15 19:28:43.540677 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.540657 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn"] Jul 15 19:28:43.555207 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.555174 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:28:43.684698 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.684661 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tmp-dir\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.684888 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.684732 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-model-cache\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.684888 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.684829 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlxwt\" (UniqueName: \"kubernetes.io/projected/1f954a44-bb37-41d0-af5f-35fd010d5b7e-kube-api-access-tlxwt\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.685004 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.684904 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tls-certs\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.685004 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.684935 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-home\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.685004 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.684959 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-dshm\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.786326 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.786248 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlxwt\" (UniqueName: \"kubernetes.io/projected/1f954a44-bb37-41d0-af5f-35fd010d5b7e-kube-api-access-tlxwt\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.786326 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.786299 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tls-certs\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.786326 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.786326 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-home\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.786626 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.786342 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-dshm\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.786626 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.786365 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tmp-dir\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.786626 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.786379 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-model-cache\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.786818 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.786780 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-home\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.786877 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.786818 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-model-cache\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.786915 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.786871 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tmp-dir\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.788719 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.788703 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-dshm\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.789016 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.789001 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tls-certs\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.793663 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.793644 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlxwt\" (UniqueName: \"kubernetes.io/projected/1f954a44-bb37-41d0-af5f-35fd010d5b7e-kube-api-access-tlxwt\") pod \"test-model-1dfe1f-kserve-7cd9f949b7-tbbfn\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.840343 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.840318 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:43.975358 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:43.975315 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn"] Jul 15 19:28:43.984154 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:28:43.984113 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f954a44_bb37_41d0_af5f_35fd010d5b7e.slice/crio-789287c2e4474501ed928a1576570bb20acb575bb96035858a6c972df545037b WatchSource:0}: Error finding container 789287c2e4474501ed928a1576570bb20acb575bb96035858a6c972df545037b: Status 404 returned error can't find the container with id 789287c2e4474501ed928a1576570bb20acb575bb96035858a6c972df545037b Jul 15 19:28:44.279733 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:44.279696 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" event={"ID":"1f954a44-bb37-41d0-af5f-35fd010d5b7e","Type":"ContainerStarted","Data":"81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2"} Jul 15 19:28:44.279916 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:44.279741 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" event={"ID":"1f954a44-bb37-41d0-af5f-35fd010d5b7e","Type":"ContainerStarted","Data":"789287c2e4474501ed928a1576570bb20acb575bb96035858a6c972df545037b"} Jul 15 19:28:44.299132 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:44.299090 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" podStartSLOduration=1.299075986 podStartE2EDuration="1.299075986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:44.298058202 +0000 UTC m=+2274.645770504" watchObservedRunningTime="2026-07-15 19:28:44.299075986 +0000 UTC m=+2274.646788285" Jul 15 19:28:53.841286 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:53.841246 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:53.841681 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:53.841300 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:53.854185 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:53.854163 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:28:54.345485 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:28:54.345449 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:29:03.622871 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:03.622836 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7"] Jul 15 19:29:03.654135 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:03.654102 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:04.682272 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:04.682239 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn"] Jul 15 19:29:04.705503 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:04.705472 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:05.476764 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:05.476732 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:05.575913 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:05.575873 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:15.268614 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:15.268566 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:16.455685 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:16.455653 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:17.738630 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:17.738598 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:18.643219 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:18.643183 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" podUID="525298c4-d14e-4393-ae35-ad35d10852e5" containerName="main" containerID="cri-o://d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001" gracePeriod=45 Jul 15 19:29:18.884586 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:18.884561 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:29:18.999045 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:18.998958 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/525298c4-d14e-4393-ae35-ad35d10852e5-tls-certs\") pod \"525298c4-d14e-4393-ae35-ad35d10852e5\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " Jul 15 19:29:18.999045 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:18.998998 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7rms\" (UniqueName: \"kubernetes.io/projected/525298c4-d14e-4393-ae35-ad35d10852e5-kube-api-access-h7rms\") pod \"525298c4-d14e-4393-ae35-ad35d10852e5\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " Jul 15 19:29:18.999260 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:18.999046 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-dshm\") pod \"525298c4-d14e-4393-ae35-ad35d10852e5\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " Jul 15 19:29:18.999260 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:18.999145 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-home\") pod \"525298c4-d14e-4393-ae35-ad35d10852e5\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " Jul 15 19:29:18.999260 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:18.999178 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-tmp-dir\") pod \"525298c4-d14e-4393-ae35-ad35d10852e5\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " Jul 15 19:29:18.999260 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:18.999202 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-model-cache\") pod \"525298c4-d14e-4393-ae35-ad35d10852e5\" (UID: \"525298c4-d14e-4393-ae35-ad35d10852e5\") " Jul 15 19:29:18.999577 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:18.999522 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-home" (OuterVolumeSpecName: "home") pod "525298c4-d14e-4393-ae35-ad35d10852e5" (UID: "525298c4-d14e-4393-ae35-ad35d10852e5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:18.999715 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:18.999587 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-model-cache" (OuterVolumeSpecName: "model-cache") pod "525298c4-d14e-4393-ae35-ad35d10852e5" (UID: "525298c4-d14e-4393-ae35-ad35d10852e5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:18.999715 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:18.999624 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "525298c4-d14e-4393-ae35-ad35d10852e5" (UID: "525298c4-d14e-4393-ae35-ad35d10852e5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:19.001266 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.001241 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-dshm" (OuterVolumeSpecName: "dshm") pod "525298c4-d14e-4393-ae35-ad35d10852e5" (UID: "525298c4-d14e-4393-ae35-ad35d10852e5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:19.001380 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.001303 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/525298c4-d14e-4393-ae35-ad35d10852e5-kube-api-access-h7rms" (OuterVolumeSpecName: "kube-api-access-h7rms") pod "525298c4-d14e-4393-ae35-ad35d10852e5" (UID: "525298c4-d14e-4393-ae35-ad35d10852e5"). InnerVolumeSpecName "kube-api-access-h7rms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:29:19.001380 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.001322 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/525298c4-d14e-4393-ae35-ad35d10852e5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "525298c4-d14e-4393-ae35-ad35d10852e5" (UID: "525298c4-d14e-4393-ae35-ad35d10852e5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:29:19.100164 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.100133 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-home\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:19.100164 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.100157 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-tmp-dir\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:19.100164 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.100167 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-model-cache\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:19.100164 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.100175 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/525298c4-d14e-4393-ae35-ad35d10852e5-tls-certs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:19.100404 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.100184 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7rms\" (UniqueName: \"kubernetes.io/projected/525298c4-d14e-4393-ae35-ad35d10852e5-kube-api-access-h7rms\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:19.100404 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.100191 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/525298c4-d14e-4393-ae35-ad35d10852e5-dshm\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:19.464988 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.464953 2574 generic.go:356] "Generic (PLEG): container finished" podID="525298c4-d14e-4393-ae35-ad35d10852e5" containerID="d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001" exitCode=0 Jul 15 19:29:19.465153 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.465006 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" Jul 15 19:29:19.465153 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.465043 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" event={"ID":"525298c4-d14e-4393-ae35-ad35d10852e5","Type":"ContainerDied","Data":"d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001"} Jul 15 19:29:19.465153 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.465090 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7" event={"ID":"525298c4-d14e-4393-ae35-ad35d10852e5","Type":"ContainerDied","Data":"44a784ce6f3290c4079df0eafef0661142f84e7b6805ef06bc3fabadc1cb98d5"} Jul 15 19:29:19.465153 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.465109 2574 scope.go:117] "RemoveContainer" containerID="d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001" Jul 15 19:29:19.474142 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.474127 2574 scope.go:117] "RemoveContainer" containerID="d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001" Jul 15 19:29:19.474439 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:29:19.474419 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001\": container with ID starting with d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001 not found: ID does not exist" containerID="d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001" Jul 15 19:29:19.474482 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.474449 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001"} err="failed to get container status \"d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001\": rpc error: code = NotFound desc = could not find container \"d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001\": container with ID starting with d58fda14c4e8baf41ee2d260c3a8319a10443394f346ca5583f0cd4484411001 not found: ID does not exist" Jul 15 19:29:19.499161 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.499134 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7"] Jul 15 19:29:19.504278 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.504256 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-7b6049/test-model-7b6049-kserve-56c78556bf-q9cv7"] Jul 15 19:29:19.702301 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.702257 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" podUID="1f954a44-bb37-41d0-af5f-35fd010d5b7e" containerName="main" containerID="cri-o://81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2" gracePeriod=45 Jul 15 19:29:19.905700 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.905668 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:19.957730 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:19.957708 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:29:20.013972 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.013946 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-model-cache\") pod \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " Jul 15 19:29:20.014155 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.014000 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tls-certs\") pod \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " Jul 15 19:29:20.014155 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.014047 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tmp-dir\") pod \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " Jul 15 19:29:20.014155 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.014148 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-dshm\") pod \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " Jul 15 19:29:20.014320 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.014188 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlxwt\" (UniqueName: \"kubernetes.io/projected/1f954a44-bb37-41d0-af5f-35fd010d5b7e-kube-api-access-tlxwt\") pod \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " Jul 15 19:29:20.014320 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.014201 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-model-cache" (OuterVolumeSpecName: "model-cache") pod "1f954a44-bb37-41d0-af5f-35fd010d5b7e" (UID: "1f954a44-bb37-41d0-af5f-35fd010d5b7e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:20.014320 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.014217 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-home\") pod \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\" (UID: \"1f954a44-bb37-41d0-af5f-35fd010d5b7e\") " Jul 15 19:29:20.014466 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.014346 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1f954a44-bb37-41d0-af5f-35fd010d5b7e" (UID: "1f954a44-bb37-41d0-af5f-35fd010d5b7e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:20.014524 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.014488 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-model-cache\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:20.014524 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.014502 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-home" (OuterVolumeSpecName: "home") pod "1f954a44-bb37-41d0-af5f-35fd010d5b7e" (UID: "1f954a44-bb37-41d0-af5f-35fd010d5b7e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:20.014608 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.014507 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tmp-dir\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:20.016271 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.016250 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-dshm" (OuterVolumeSpecName: "dshm") pod "1f954a44-bb37-41d0-af5f-35fd010d5b7e" (UID: "1f954a44-bb37-41d0-af5f-35fd010d5b7e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:20.016383 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.016341 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1f954a44-bb37-41d0-af5f-35fd010d5b7e" (UID: "1f954a44-bb37-41d0-af5f-35fd010d5b7e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:29:20.016383 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.016362 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f954a44-bb37-41d0-af5f-35fd010d5b7e-kube-api-access-tlxwt" (OuterVolumeSpecName: "kube-api-access-tlxwt") pod "1f954a44-bb37-41d0-af5f-35fd010d5b7e" (UID: "1f954a44-bb37-41d0-af5f-35fd010d5b7e"). InnerVolumeSpecName "kube-api-access-tlxwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:29:20.115379 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.115349 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-dshm\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:20.115379 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.115374 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlxwt\" (UniqueName: \"kubernetes.io/projected/1f954a44-bb37-41d0-af5f-35fd010d5b7e-kube-api-access-tlxwt\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:20.115379 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.115384 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f954a44-bb37-41d0-af5f-35fd010d5b7e-home\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:20.115592 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.115394 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f954a44-bb37-41d0-af5f-35fd010d5b7e-tls-certs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:29:20.207729 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.207698 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="525298c4-d14e-4393-ae35-ad35d10852e5" path="/var/lib/kubelet/pods/525298c4-d14e-4393-ae35-ad35d10852e5/volumes" Jul 15 19:29:20.471271 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.471185 2574 generic.go:356] "Generic (PLEG): container finished" podID="1f954a44-bb37-41d0-af5f-35fd010d5b7e" containerID="81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2" exitCode=0 Jul 15 19:29:20.471271 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.471226 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" Jul 15 19:29:20.471271 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.471265 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" event={"ID":"1f954a44-bb37-41d0-af5f-35fd010d5b7e","Type":"ContainerDied","Data":"81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2"} Jul 15 19:29:20.471547 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.471301 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn" event={"ID":"1f954a44-bb37-41d0-af5f-35fd010d5b7e","Type":"ContainerDied","Data":"789287c2e4474501ed928a1576570bb20acb575bb96035858a6c972df545037b"} Jul 15 19:29:20.471547 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.471322 2574 scope.go:117] "RemoveContainer" containerID="81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2" Jul 15 19:29:20.479779 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.479743 2574 scope.go:117] "RemoveContainer" containerID="81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2" Jul 15 19:29:20.480067 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:29:20.480040 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2\": container with ID starting with 81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2 not found: ID does not exist" containerID="81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2" Jul 15 19:29:20.480147 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.480072 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2"} err="failed to get container status \"81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2\": rpc error: code = NotFound desc = could not find container \"81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2\": container with ID starting with 81d18a8a4490c513ed80c02ed4f55b3bda4ba19fcc520f0ea643ac2ce529e9e2 not found: ID does not exist" Jul 15 19:29:20.487071 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.487048 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn"] Jul 15 19:29:20.491459 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:20.491437 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-1dfe1f/test-model-1dfe1f-kserve-7cd9f949b7-tbbfn"] Jul 15 19:29:22.206745 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:22.206711 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f954a44-bb37-41d0-af5f-35fd010d5b7e" path="/var/lib/kubelet/pods/1f954a44-bb37-41d0-af5f-35fd010d5b7e/volumes" Jul 15 19:29:25.009875 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:25.009846 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:25.111037 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:25.111002 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:25.780635 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:25.780579 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:25.854363 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:25.854330 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:30.613689 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:30.613653 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:30.854394 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:30.854363 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:31.504939 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:31.504903 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:31.609433 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:31.609404 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:41.677833 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:41.677800 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:46.114900 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:46.114816 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:51.959072 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:51.959039 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:52.118444 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:52.118410 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:56.767099 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:56.767069 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:57.011827 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:57.011769 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:57.353205 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:57.353165 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:57.656064 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:57.655988 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:29:57.980684 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:29:57.980604 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:00.141135 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.141100 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735730-vpkwd"] Jul 15 19:30:00.141530 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.141439 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f954a44-bb37-41d0-af5f-35fd010d5b7e" containerName="main" Jul 15 19:30:00.141530 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.141449 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f954a44-bb37-41d0-af5f-35fd010d5b7e" containerName="main" Jul 15 19:30:00.141530 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.141460 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="525298c4-d14e-4393-ae35-ad35d10852e5" containerName="main" Jul 15 19:30:00.141530 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.141465 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="525298c4-d14e-4393-ae35-ad35d10852e5" containerName="main" Jul 15 19:30:00.141530 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.141524 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="525298c4-d14e-4393-ae35-ad35d10852e5" containerName="main" Jul 15 19:30:00.141530 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.141531 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f954a44-bb37-41d0-af5f-35fd010d5b7e" containerName="main" Jul 15 19:30:00.144518 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.144500 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-vpkwd" Jul 15 19:30:00.147386 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.147364 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2f4w7\"" Jul 15 19:30:00.153561 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.153536 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735730-vpkwd"] Jul 15 19:30:00.155014 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.154992 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-vpkwd" Jul 15 19:30:00.291797 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.291754 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735730-vpkwd"] Jul 15 19:30:00.293417 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:30:00.293389 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b31560_730c_4d14_9146_b731d1edf91e.slice/crio-b79a47bbdbdf650749b9d28967e9b5d1213cf47aa489081d154e9eba21f95a86 WatchSource:0}: Error finding container b79a47bbdbdf650749b9d28967e9b5d1213cf47aa489081d154e9eba21f95a86: Status 404 returned error can't find the container with id b79a47bbdbdf650749b9d28967e9b5d1213cf47aa489081d154e9eba21f95a86 Jul 15 19:30:00.295261 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.295241 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:30:00.420813 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.420696 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:00.522737 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.522701 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:00.601877 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.601834 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:00.679031 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.678945 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-vpkwd" event={"ID":"96b31560-730c-4d14-9146-b731d1edf91e","Type":"ContainerStarted","Data":"b79a47bbdbdf650749b9d28967e9b5d1213cf47aa489081d154e9eba21f95a86"} Jul 15 19:30:00.702332 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:00.702300 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:01.115183 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:01.115148 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:01.214872 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:01.214843 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:03.697880 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:03.697851 2574 generic.go:356] "Generic (PLEG): container finished" podID="96b31560-730c-4d14-9146-b731d1edf91e" containerID="62245d14d75c00c29e1241d9cbf6b29c3b2f52362f8607eaee570d400dc24d73" exitCode=0 Jul 15 19:30:03.698242 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:03.697931 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-vpkwd" event={"ID":"96b31560-730c-4d14-9146-b731d1edf91e","Type":"ContainerDied","Data":"62245d14d75c00c29e1241d9cbf6b29c3b2f52362f8607eaee570d400dc24d73"} Jul 15 19:30:04.833641 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:04.833613 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-vpkwd" Jul 15 19:30:04.949308 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:04.949279 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:05.401004 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:05.400972 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:05.710589 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:05.710517 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-vpkwd" Jul 15 19:30:05.710589 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:05.710539 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-vpkwd" event={"ID":"96b31560-730c-4d14-9146-b731d1edf91e","Type":"ContainerDied","Data":"b79a47bbdbdf650749b9d28967e9b5d1213cf47aa489081d154e9eba21f95a86"} Jul 15 19:30:05.710589 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:05.710582 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b79a47bbdbdf650749b9d28967e9b5d1213cf47aa489081d154e9eba21f95a86" Jul 15 19:30:05.842390 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:05.842355 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:06.234946 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:06.234917 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:08.566011 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:08.565970 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:08.661588 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:08.661558 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:08.749465 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:08.749437 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:09.088848 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:09.088817 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:10.114834 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:10.114776 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:10.215336 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:10.215303 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:13.016999 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:13.016961 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:21.779444 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:21.779372 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:21.879386 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:21.879353 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:28.303485 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:28.303452 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:28.404480 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:28.404448 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:30.020075 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:30.020037 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:30.120582 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:30.120551 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:31.781673 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:31.781641 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:31.883099 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:31.883065 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:32.033387 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:32.033298 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:36.270993 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:36.270963 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:42.499324 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:42.499290 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:42.566392 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:42.566359 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:47.381758 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.381724 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt"] Jul 15 19:30:47.382403 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.382382 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96b31560-730c-4d14-9146-b731d1edf91e" containerName="cleanup" Jul 15 19:30:47.382495 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.382405 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b31560-730c-4d14-9146-b731d1edf91e" containerName="cleanup" Jul 15 19:30:47.382558 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.382513 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="96b31560-730c-4d14-9146-b731d1edf91e" containerName="cleanup" Jul 15 19:30:47.389028 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.389002 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt" Jul 15 19:30:47.391684 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.391657 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2f4w7\"" Jul 15 19:30:47.392061 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.391678 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 19:30:47.393700 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.393677 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt"] Jul 15 19:30:47.461703 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.461666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a8702ca-9d4a-4249-a49c-086e98395ecc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt\" (UID: \"7a8702ca-9d4a-4249-a49c-086e98395ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt" Jul 15 19:30:47.463216 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.463177 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:47.562706 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.562673 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a8702ca-9d4a-4249-a49c-086e98395ecc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt\" (UID: \"7a8702ca-9d4a-4249-a49c-086e98395ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt" Jul 15 19:30:47.563273 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.563253 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a8702ca-9d4a-4249-a49c-086e98395ecc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt\" (UID: \"7a8702ca-9d4a-4249-a49c-086e98395ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt" Jul 15 19:30:47.563916 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.563894 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:47.700925 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.700829 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt" Jul 15 19:30:47.836890 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.836856 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt"] Jul 15 19:30:47.838932 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:30:47.838906 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a8702ca_9d4a_4249_a49c_086e98395ecc.slice/crio-eda2fee419daa2c944db1acd467b610942c6b47f1a4a56a457578b97328c95ff WatchSource:0}: Error finding container eda2fee419daa2c944db1acd467b610942c6b47f1a4a56a457578b97328c95ff: Status 404 returned error can't find the container with id eda2fee419daa2c944db1acd467b610942c6b47f1a4a56a457578b97328c95ff Jul 15 19:30:47.896892 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.896852 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:47.935374 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.935341 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt" event={"ID":"7a8702ca-9d4a-4249-a49c-086e98395ecc","Type":"ContainerStarted","Data":"eda2fee419daa2c944db1acd467b610942c6b47f1a4a56a457578b97328c95ff"} Jul 15 19:30:47.997596 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:47.997512 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:30:48.942848 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:48.942745 2574 generic.go:356] "Generic (PLEG): container finished" podID="7a8702ca-9d4a-4249-a49c-086e98395ecc" containerID="1dd1b921bdcc7fd48388061495972111c4e824cebfb65786ec23520f20e42231" exitCode=0 Jul 15 19:30:48.942848 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:48.942815 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt" event={"ID":"7a8702ca-9d4a-4249-a49c-086e98395ecc","Type":"ContainerDied","Data":"1dd1b921bdcc7fd48388061495972111c4e824cebfb65786ec23520f20e42231"} Jul 15 19:30:50.075077 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:50.075053 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt" Jul 15 19:30:50.081071 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:50.081052 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a8702ca-9d4a-4249-a49c-086e98395ecc-maas-api-service-ca\") pod \"7a8702ca-9d4a-4249-a49c-086e98395ecc\" (UID: \"7a8702ca-9d4a-4249-a49c-086e98395ecc\") " Jul 15 19:30:50.081380 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:50.081359 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a8702ca-9d4a-4249-a49c-086e98395ecc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7a8702ca-9d4a-4249-a49c-086e98395ecc" (UID: "7a8702ca-9d4a-4249-a49c-086e98395ecc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:30:50.181928 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:50.181886 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a8702ca-9d4a-4249-a49c-086e98395ecc-maas-api-service-ca\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:30:50.339610 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:50.339587 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:30:50.347071 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:50.347048 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:30:50.954636 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:50.954599 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt" event={"ID":"7a8702ca-9d4a-4249-a49c-086e98395ecc","Type":"ContainerDied","Data":"eda2fee419daa2c944db1acd467b610942c6b47f1a4a56a457578b97328c95ff"} Jul 15 19:30:50.954636 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:50.954626 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt" Jul 15 19:30:50.954862 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:50.954653 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eda2fee419daa2c944db1acd467b610942c6b47f1a4a56a457578b97328c95ff" Jul 15 19:30:57.996162 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:30:57.996125 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:02.469938 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:02.469895 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:08.627873 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:08.627833 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:08.735181 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:08.735147 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:11.876317 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:11.876258 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:13.191403 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:13.191323 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:13.423948 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:13.423912 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:13.780263 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:13.780225 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:14.148503 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:14.148421 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:14.413353 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:14.413262 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:16.908664 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:16.908612 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:17.018581 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:17.018546 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:17.238114 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:17.238012 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:17.482387 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:17.482342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:17.998516 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:17.998470 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:18.091043 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:18.091008 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:21.448908 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:21.448876 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:21.793249 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:21.793216 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:22.246329 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:22.246232 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:22.457004 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:22.456967 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:24.629873 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:24.629836 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:24.996857 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:24.996705 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:25.110646 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:25.110609 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:25.201425 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:25.201389 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:25.312140 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:25.312094 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:26.342631 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:26.342567 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:26.442629 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:26.442596 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:29.513385 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:29.513346 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:38.802726 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:38.802691 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:38.909998 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:38.909960 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:41.922089 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.922052 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9"] Jul 15 19:31:41.922576 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.922551 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a8702ca-9d4a-4249-a49c-086e98395ecc" containerName="revoke-keys" Jul 15 19:31:41.922576 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.922570 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a8702ca-9d4a-4249-a49c-086e98395ecc" containerName="revoke-keys" Jul 15 19:31:41.922704 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.922689 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a8702ca-9d4a-4249-a49c-086e98395ecc" containerName="revoke-keys" Jul 15 19:31:41.931775 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.931734 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:41.935228 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.935193 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-257300\"/\"kube-root-ca.crt\"" Jul 15 19:31:41.935392 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.935256 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-257300\"/\"auth-test-model-257300-kserve-self-signed-certs\"" Jul 15 19:31:41.935392 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.935316 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-257300\"/\"default-dockercfg-btkmq\"" Jul 15 19:31:41.935511 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.935192 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-257300\"/\"openshift-service-ca.crt\"" Jul 15 19:31:41.936516 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.936443 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9"] Jul 15 19:31:41.936645 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.936532 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-model-cache\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:41.936645 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.936588 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-dshm\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:41.936645 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.936622 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlgm5\" (UniqueName: \"kubernetes.io/projected/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-kube-api-access-hlgm5\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:41.937232 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.936659 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-home\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:41.937232 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.936695 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tmp-dir\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:41.937232 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.936831 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tls-certs\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:41.960673 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:41.960637 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:42.037218 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.037180 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlgm5\" (UniqueName: \"kubernetes.io/projected/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-kube-api-access-hlgm5\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.037218 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.037220 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-home\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.037453 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.037250 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tmp-dir\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.037453 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.037303 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tls-certs\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.037453 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.037353 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-model-cache\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.037453 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.037382 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-dshm\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.037705 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.037683 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-home\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.037778 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.037701 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tmp-dir\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.037886 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.037869 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-model-cache\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.039676 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.039656 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-dshm\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.040113 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.040090 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tls-certs\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.045439 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.045408 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlgm5\" (UniqueName: \"kubernetes.io/projected/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-kube-api-access-hlgm5\") pod \"auth-test-model-257300-kserve-7c55b6d5ff-922z9\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.244924 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.244823 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:42.383610 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:42.383584 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9"] Jul 15 19:31:42.385587 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:31:42.385559 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7d7b3f0_86e7_4201_b7fb_8c5a373b96e8.slice/crio-4e80384c097e71dc7167f9cdf3c6941c5b5cfcfe8d3ae7e2cf00061a70a10d16 WatchSource:0}: Error finding container 4e80384c097e71dc7167f9cdf3c6941c5b5cfcfe8d3ae7e2cf00061a70a10d16: Status 404 returned error can't find the container with id 4e80384c097e71dc7167f9cdf3c6941c5b5cfcfe8d3ae7e2cf00061a70a10d16 Jul 15 19:31:43.237712 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:43.237678 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" event={"ID":"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8","Type":"ContainerStarted","Data":"db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f"} Jul 15 19:31:43.237712 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:43.237719 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" event={"ID":"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8","Type":"ContainerStarted","Data":"4e80384c097e71dc7167f9cdf3c6941c5b5cfcfe8d3ae7e2cf00061a70a10d16"} Jul 15 19:31:43.256330 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:43.256276 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" podStartSLOduration=2.256259489 podStartE2EDuration="2.256259489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:31:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:31:43.255290448 +0000 UTC m=+2453.603002761" watchObservedRunningTime="2026-07-15 19:31:43.256259489 +0000 UTC m=+2453.603971829" Jul 15 19:31:46.610247 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:46.610210 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:46.713431 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:46.713395 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:50.762612 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:50.762578 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:31:52.245674 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:52.245635 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:52.245674 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:52.245680 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:52.258464 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:52.258435 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:31:52.306238 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:31:52.306205 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:32:06.736534 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.736492 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt"] Jul 15 19:32:06.740136 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.740112 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.743525 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.743503 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"openshift-service-ca.crt\"" Jul 15 19:32:06.743676 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.743532 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"kube-root-ca.crt\"" Jul 15 19:32:06.743676 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.743616 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"auth-test-model-0dc297-kserve-self-signed-certs\"" Jul 15 19:32:06.743676 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.743616 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"default-dockercfg-2g9wq\"" Jul 15 19:32:06.749509 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.749488 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt"] Jul 15 19:32:06.850075 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.850041 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:32:06.854189 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.854161 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tclnf\" (UniqueName: \"kubernetes.io/projected/c8f05155-8fb4-4913-9227-71f01b93b50b-kube-api-access-tclnf\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.854319 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.854222 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8f05155-8fb4-4913-9227-71f01b93b50b-tls-certs\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.854319 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.854280 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-dshm\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.854404 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.854325 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-home\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.854404 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.854344 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-tmp-dir\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.854404 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.854386 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-model-cache\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.955561 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.955521 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8f05155-8fb4-4913-9227-71f01b93b50b-tls-certs\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.955561 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.955562 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-dshm\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.955842 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.955597 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-home\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.955842 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.955614 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-tmp-dir\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.955842 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.955635 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-model-cache\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.955842 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.955668 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tclnf\" (UniqueName: \"kubernetes.io/projected/c8f05155-8fb4-4913-9227-71f01b93b50b-kube-api-access-tclnf\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.956119 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.956080 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-tmp-dir\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.956182 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.956098 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-home\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.956182 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.956155 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-model-cache\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.957986 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.957956 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-dshm\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.958272 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.958249 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8f05155-8fb4-4913-9227-71f01b93b50b-tls-certs\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:06.970649 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:06.970624 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tclnf\" (UniqueName: \"kubernetes.io/projected/c8f05155-8fb4-4913-9227-71f01b93b50b-kube-api-access-tclnf\") pod \"auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:07.050406 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:07.050365 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:07.191733 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:07.191702 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt"] Jul 15 19:32:07.193261 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:32:07.193237 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8f05155_8fb4_4913_9227_71f01b93b50b.slice/crio-2a3dbfd4ba3393952c279e1014a5bcf58ffa72041f1dd6da9e88d0c187711257 WatchSource:0}: Error finding container 2a3dbfd4ba3393952c279e1014a5bcf58ffa72041f1dd6da9e88d0c187711257: Status 404 returned error can't find the container with id 2a3dbfd4ba3393952c279e1014a5bcf58ffa72041f1dd6da9e88d0c187711257 Jul 15 19:32:07.375454 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:07.375356 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" event={"ID":"c8f05155-8fb4-4913-9227-71f01b93b50b","Type":"ContainerStarted","Data":"c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615"} Jul 15 19:32:07.375454 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:07.375401 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" event={"ID":"c8f05155-8fb4-4913-9227-71f01b93b50b","Type":"ContainerStarted","Data":"2a3dbfd4ba3393952c279e1014a5bcf58ffa72041f1dd6da9e88d0c187711257"} Jul 15 19:32:07.392669 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:07.392617 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" podStartSLOduration=1.392599094 podStartE2EDuration="1.392599094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:32:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:32:07.391898004 +0000 UTC m=+2477.739610323" watchObservedRunningTime="2026-07-15 19:32:07.392599094 +0000 UTC m=+2477.740311439" Jul 15 19:32:17.051216 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:17.051174 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:17.051216 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:17.051230 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:17.063942 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:17.063912 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:32:17.438616 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:32:17.438534 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:33:04.039481 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:33:04.039449 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:33:05.955110 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:33:05.955077 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:34:14.385830 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:14.385727 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9"] Jul 15 19:34:14.406153 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:14.406118 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:34:15.238550 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:15.238510 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt"] Jul 15 19:34:15.273097 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:15.273043 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:34:17.456992 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:17.456937 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:34:17.559011 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:17.558975 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:34:26.160028 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:26.159984 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:34:29.406200 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.406137 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" podUID="d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" containerName="main" containerID="cri-o://db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f" gracePeriod=45 Jul 15 19:34:29.660163 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.660091 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:34:29.738740 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.738705 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlgm5\" (UniqueName: \"kubernetes.io/projected/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-kube-api-access-hlgm5\") pod \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " Jul 15 19:34:29.738939 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.738759 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-home\") pod \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " Jul 15 19:34:29.738939 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.738810 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tmp-dir\") pod \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " Jul 15 19:34:29.739043 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.738954 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tls-certs\") pod \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " Jul 15 19:34:29.739043 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.738991 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-dshm\") pod \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " Jul 15 19:34:29.739151 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.739062 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-model-cache\") pod \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\" (UID: \"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8\") " Jul 15 19:34:29.739151 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.739072 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" (UID: "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:34:29.739151 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.739067 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-home" (OuterVolumeSpecName: "home") pod "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" (UID: "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:34:29.739334 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.739316 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-home\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:29.739379 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.739339 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tmp-dir\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:29.739420 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.739378 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-model-cache" (OuterVolumeSpecName: "model-cache") pod "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" (UID: "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:34:29.741173 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.741148 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-dshm" (OuterVolumeSpecName: "dshm") pod "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" (UID: "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:34:29.741481 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.741449 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-kube-api-access-hlgm5" (OuterVolumeSpecName: "kube-api-access-hlgm5") pod "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" (UID: "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8"). InnerVolumeSpecName "kube-api-access-hlgm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:34:29.741585 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.741451 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" (UID: "d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:34:29.839877 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.839842 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-tls-certs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:29.839877 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.839872 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-dshm\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:29.839877 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.839880 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-model-cache\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:29.839877 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:29.839888 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlgm5\" (UniqueName: \"kubernetes.io/projected/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8-kube-api-access-hlgm5\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:30.125817 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.125768 2574 generic.go:356] "Generic (PLEG): container finished" podID="d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" containerID="db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f" exitCode=0 Jul 15 19:34:30.125990 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.125853 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" Jul 15 19:34:30.125990 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.125852 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" event={"ID":"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8","Type":"ContainerDied","Data":"db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f"} Jul 15 19:34:30.125990 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.125909 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9" event={"ID":"d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8","Type":"ContainerDied","Data":"4e80384c097e71dc7167f9cdf3c6941c5b5cfcfe8d3ae7e2cf00061a70a10d16"} Jul 15 19:34:30.125990 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.125924 2574 scope.go:117] "RemoveContainer" containerID="db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f" Jul 15 19:34:30.135063 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.135040 2574 scope.go:117] "RemoveContainer" containerID="db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f" Jul 15 19:34:30.135368 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:34:30.135347 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f\": container with ID starting with db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f not found: ID does not exist" containerID="db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f" Jul 15 19:34:30.135467 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.135375 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f"} err="failed to get container status \"db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f\": rpc error: code = NotFound desc = could not find container \"db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f\": container with ID starting with db95ef4ffd5736bcfc9912d8116391dfda5624e0c80e83624fda0f52209ace5f not found: ID does not exist" Jul 15 19:34:30.146987 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.146941 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9"] Jul 15 19:34:30.150643 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.150616 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-257300/auth-test-model-257300-kserve-7c55b6d5ff-922z9"] Jul 15 19:34:30.207714 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.207679 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" path="/var/lib/kubelet/pods/d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8/volumes" Jul 15 19:34:30.259144 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.259105 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" podUID="c8f05155-8fb4-4913-9227-71f01b93b50b" containerName="main" containerID="cri-o://c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615" gracePeriod=45 Jul 15 19:34:30.513225 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.513199 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:34:30.547728 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.547695 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-home\") pod \"c8f05155-8fb4-4913-9227-71f01b93b50b\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " Jul 15 19:34:30.547728 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.547731 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-tmp-dir\") pod \"c8f05155-8fb4-4913-9227-71f01b93b50b\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " Jul 15 19:34:30.548032 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.547762 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8f05155-8fb4-4913-9227-71f01b93b50b-tls-certs\") pod \"c8f05155-8fb4-4913-9227-71f01b93b50b\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " Jul 15 19:34:30.548032 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.547813 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-dshm\") pod \"c8f05155-8fb4-4913-9227-71f01b93b50b\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " Jul 15 19:34:30.548032 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.547894 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-model-cache\") pod \"c8f05155-8fb4-4913-9227-71f01b93b50b\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " Jul 15 19:34:30.548032 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.547923 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tclnf\" (UniqueName: \"kubernetes.io/projected/c8f05155-8fb4-4913-9227-71f01b93b50b-kube-api-access-tclnf\") pod \"c8f05155-8fb4-4913-9227-71f01b93b50b\" (UID: \"c8f05155-8fb4-4913-9227-71f01b93b50b\") " Jul 15 19:34:30.548264 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.548056 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c8f05155-8fb4-4913-9227-71f01b93b50b" (UID: "c8f05155-8fb4-4913-9227-71f01b93b50b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:34:30.548264 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.548072 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-home" (OuterVolumeSpecName: "home") pod "c8f05155-8fb4-4913-9227-71f01b93b50b" (UID: "c8f05155-8fb4-4913-9227-71f01b93b50b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:34:30.548264 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.548169 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-model-cache" (OuterVolumeSpecName: "model-cache") pod "c8f05155-8fb4-4913-9227-71f01b93b50b" (UID: "c8f05155-8fb4-4913-9227-71f01b93b50b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:34:30.548438 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.548405 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-model-cache\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:30.548438 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.548429 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-home\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:30.548545 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.548443 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-tmp-dir\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:30.550173 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.550137 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8f05155-8fb4-4913-9227-71f01b93b50b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c8f05155-8fb4-4913-9227-71f01b93b50b" (UID: "c8f05155-8fb4-4913-9227-71f01b93b50b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:34:30.550319 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.550209 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8f05155-8fb4-4913-9227-71f01b93b50b-kube-api-access-tclnf" (OuterVolumeSpecName: "kube-api-access-tclnf") pod "c8f05155-8fb4-4913-9227-71f01b93b50b" (UID: "c8f05155-8fb4-4913-9227-71f01b93b50b"). InnerVolumeSpecName "kube-api-access-tclnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:34:30.550319 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.550216 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-dshm" (OuterVolumeSpecName: "dshm") pod "c8f05155-8fb4-4913-9227-71f01b93b50b" (UID: "c8f05155-8fb4-4913-9227-71f01b93b50b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:34:30.649337 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.649240 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8f05155-8fb4-4913-9227-71f01b93b50b-tls-certs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:30.649337 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.649270 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8f05155-8fb4-4913-9227-71f01b93b50b-dshm\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:30.649337 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:30.649281 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tclnf\" (UniqueName: \"kubernetes.io/projected/c8f05155-8fb4-4913-9227-71f01b93b50b-kube-api-access-tclnf\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:34:31.133089 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:31.133050 2574 generic.go:356] "Generic (PLEG): container finished" podID="c8f05155-8fb4-4913-9227-71f01b93b50b" containerID="c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615" exitCode=0 Jul 15 19:34:31.133268 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:31.133094 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" Jul 15 19:34:31.133268 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:31.133140 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" event={"ID":"c8f05155-8fb4-4913-9227-71f01b93b50b","Type":"ContainerDied","Data":"c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615"} Jul 15 19:34:31.133268 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:31.133183 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt" event={"ID":"c8f05155-8fb4-4913-9227-71f01b93b50b","Type":"ContainerDied","Data":"2a3dbfd4ba3393952c279e1014a5bcf58ffa72041f1dd6da9e88d0c187711257"} Jul 15 19:34:31.133268 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:31.133204 2574 scope.go:117] "RemoveContainer" containerID="c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615" Jul 15 19:34:31.142270 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:31.142249 2574 scope.go:117] "RemoveContainer" containerID="c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615" Jul 15 19:34:31.142556 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:34:31.142537 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615\": container with ID starting with c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615 not found: ID does not exist" containerID="c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615" Jul 15 19:34:31.142595 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:31.142566 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615"} err="failed to get container status \"c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615\": rpc error: code = NotFound desc = could not find container \"c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615\": container with ID starting with c1b2c6b091e6ca5edc2b7cb1ca84effae6c29f05732b50d8f1928e11eeffa615 not found: ID does not exist" Jul 15 19:34:31.153890 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:31.153860 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt"] Jul 15 19:34:31.157505 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:31.157480 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/auth-test-model-0dc297-kserve-6cbbfbb46c-wq2xt"] Jul 15 19:34:32.206589 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:32.206550 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8f05155-8fb4-4913-9227-71f01b93b50b" path="/var/lib/kubelet/pods/c8f05155-8fb4-4913-9227-71f01b93b50b/volumes" Jul 15 19:34:41.420176 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.420125 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd"] Jul 15 19:34:41.420575 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.420489 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8f05155-8fb4-4913-9227-71f01b93b50b" containerName="main" Jul 15 19:34:41.420575 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.420499 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8f05155-8fb4-4913-9227-71f01b93b50b" containerName="main" Jul 15 19:34:41.420575 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.420508 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" containerName="main" Jul 15 19:34:41.420575 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.420513 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" containerName="main" Jul 15 19:34:41.420711 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.420583 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8f05155-8fb4-4913-9227-71f01b93b50b" containerName="main" Jul 15 19:34:41.420711 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.420592 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7d7b3f0-86e7-4201-b7fb-8c5a373b96e8" containerName="main" Jul 15 19:34:41.424814 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.424775 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.428180 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.428154 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-257300\"/\"sub-test-model-257300-kserve-self-signed-certs\"" Jul 15 19:34:41.428359 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.428188 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-257300\"/\"default-dockercfg-btkmq\"" Jul 15 19:34:41.428359 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.428287 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-257300\"/\"kube-root-ca.crt\"" Jul 15 19:34:41.428359 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.428296 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-257300\"/\"openshift-service-ca.crt\"" Jul 15 19:34:41.434555 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.434529 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd"] Jul 15 19:34:41.544190 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.544151 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwgjv\" (UniqueName: \"kubernetes.io/projected/2d76430e-eb8e-4686-a940-31bdb6be4eb0-kube-api-access-pwgjv\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.544190 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.544199 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tls-certs\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.544478 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.544228 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-model-cache\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.544478 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.544316 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tmp-dir\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.544478 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.544378 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-home\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.544478 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.544407 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-dshm\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.645147 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.645105 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwgjv\" (UniqueName: \"kubernetes.io/projected/2d76430e-eb8e-4686-a940-31bdb6be4eb0-kube-api-access-pwgjv\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.645361 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.645156 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tls-certs\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.645361 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.645191 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-model-cache\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.645361 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.645235 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tmp-dir\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.645361 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.645272 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-home\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.645361 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.645297 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-dshm\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.645732 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.645701 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tmp-dir\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.645877 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.645750 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-model-cache\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.645877 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.645823 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-home\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.647520 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.647502 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-dshm\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.647864 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.647847 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tls-certs\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.659467 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.659443 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwgjv\" (UniqueName: \"kubernetes.io/projected/2d76430e-eb8e-4686-a940-31bdb6be4eb0-kube-api-access-pwgjv\") pod \"sub-test-model-257300-kserve-6f75f9cc77-dn5dd\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.736209 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.736107 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:41.879580 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.879551 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd"] Jul 15 19:34:41.881224 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:34:41.881196 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d76430e_eb8e_4686_a940_31bdb6be4eb0.slice/crio-160756c6ec7719e705be4fd881427fdf8e314a4e3fc8b5c8107ab11a04b5d2da WatchSource:0}: Error finding container 160756c6ec7719e705be4fd881427fdf8e314a4e3fc8b5c8107ab11a04b5d2da: Status 404 returned error can't find the container with id 160756c6ec7719e705be4fd881427fdf8e314a4e3fc8b5c8107ab11a04b5d2da Jul 15 19:34:41.921810 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:41.921766 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:34:42.196848 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:42.196778 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" event={"ID":"2d76430e-eb8e-4686-a940-31bdb6be4eb0","Type":"ContainerStarted","Data":"bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483"} Jul 15 19:34:42.196848 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:42.196848 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" event={"ID":"2d76430e-eb8e-4686-a940-31bdb6be4eb0","Type":"ContainerStarted","Data":"160756c6ec7719e705be4fd881427fdf8e314a4e3fc8b5c8107ab11a04b5d2da"} Jul 15 19:34:42.215814 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:42.215734 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" podStartSLOduration=1.215715947 podStartE2EDuration="1.215715947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:34:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:34:42.214291987 +0000 UTC m=+2632.562004287" watchObservedRunningTime="2026-07-15 19:34:42.215715947 +0000 UTC m=+2632.563428245" Jul 15 19:34:51.737105 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:51.737060 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:51.737582 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:51.737355 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:51.750065 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:51.750036 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:34:52.258291 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:34:52.258260 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:35:01.644412 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.644374 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l"] Jul 15 19:35:01.647691 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.647673 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.650133 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.650114 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"kube-root-ca.crt\"" Jul 15 19:35:01.651277 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.651259 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"default-dockercfg-2g9wq\"" Jul 15 19:35:01.651356 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.651330 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"sub-test-model-0dc297-kserve-self-signed-certs\"" Jul 15 19:35:01.651405 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.651365 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"openshift-service-ca.crt\"" Jul 15 19:35:01.657199 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.657174 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l"] Jul 15 19:35:01.672257 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.672221 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:35:01.736940 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.736902 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tmp-dir\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.737127 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.736955 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-home\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.737127 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.737041 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrh9d\" (UniqueName: \"kubernetes.io/projected/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-kube-api-access-jrh9d\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.737127 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.737075 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-model-cache\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.737127 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.737096 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tls-certs\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.737310 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.737156 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-dshm\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.838012 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.837979 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tmp-dir\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.838212 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.838026 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-home\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.838212 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.838068 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrh9d\" (UniqueName: \"kubernetes.io/projected/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-kube-api-access-jrh9d\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.838212 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.838097 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-model-cache\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.838212 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.838118 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tls-certs\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.838212 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.838162 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-dshm\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.838474 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.838435 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tmp-dir\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.838535 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.838479 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-home\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.838587 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.838525 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-model-cache\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.840509 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.840484 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-dshm\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.840641 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.840625 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tls-certs\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.848174 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.848138 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrh9d\" (UniqueName: \"kubernetes.io/projected/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-kube-api-access-jrh9d\") pod \"sub-test-model-0dc297-kserve-669cc55c5c-gwv2l\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:01.957932 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:01.957828 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:02.096558 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:02.096531 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l"] Jul 15 19:35:02.098110 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:35:02.098083 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e3ea0ef_53b9_4687_a94c_378589d5c8bf.slice/crio-318f78027ed6d3e44aa0757a3f3ededd0e31c31bbd136c2db0de41d4c06db217 WatchSource:0}: Error finding container 318f78027ed6d3e44aa0757a3f3ededd0e31c31bbd136c2db0de41d4c06db217: Status 404 returned error can't find the container with id 318f78027ed6d3e44aa0757a3f3ededd0e31c31bbd136c2db0de41d4c06db217 Jul 15 19:35:02.299207 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:02.299166 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" event={"ID":"8e3ea0ef-53b9-4687-a94c-378589d5c8bf","Type":"ContainerStarted","Data":"e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668"} Jul 15 19:35:02.299397 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:02.299215 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" event={"ID":"8e3ea0ef-53b9-4687-a94c-378589d5c8bf","Type":"ContainerStarted","Data":"318f78027ed6d3e44aa0757a3f3ededd0e31c31bbd136c2db0de41d4c06db217"} Jul 15 19:35:02.317414 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:02.317357 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" podStartSLOduration=1.317341883 podStartE2EDuration="1.317341883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:35:02.315260453 +0000 UTC m=+2652.662972743" watchObservedRunningTime="2026-07-15 19:35:02.317341883 +0000 UTC m=+2652.665054181" Jul 15 19:35:11.958733 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:11.958697 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:11.959140 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:11.958753 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:11.971403 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:11.971379 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:12.362557 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:12.362525 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:35:50.372364 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:50.372282 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:35:50.381733 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:50.381711 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cjxwn_e241c2dc-dedc-4a81-8e1b-6a5488efeddb/ovn-acl-logging/0.log" Jul 15 19:35:52.370932 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:52.370901 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd"] Jul 15 19:35:52.402970 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:52.402937 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:35:53.250391 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:53.250362 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l"] Jul 15 19:35:53.273007 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:53.272979 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:35:59.942107 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:59.942073 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c"] Jul 15 19:35:59.945307 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:59.945291 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:35:59.947842 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:59.947818 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-257300\"/\"rate-test-model-257300-kserve-self-signed-certs\"" Jul 15 19:35:59.954483 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:59.954459 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c"] Jul 15 19:35:59.975143 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:35:59.975119 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:36:00.078833 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.078775 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-model-cache\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.078988 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.078889 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6afac70c-04d9-484d-a228-7f08d87bea15-tls-certs\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.078988 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.078915 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-home\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.078988 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.078952 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-dshm\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.078988 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.078982 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpjk6\" (UniqueName: \"kubernetes.io/projected/6afac70c-04d9-484d-a228-7f08d87bea15-kube-api-access-tpjk6\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.079127 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.079033 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-tmp-dir\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.179961 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.179924 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6afac70c-04d9-484d-a228-7f08d87bea15-tls-certs\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.179961 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.179959 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-home\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.180201 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.179975 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-dshm\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.180201 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.180003 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpjk6\" (UniqueName: \"kubernetes.io/projected/6afac70c-04d9-484d-a228-7f08d87bea15-kube-api-access-tpjk6\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.180201 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.180041 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-tmp-dir\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.180201 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.180107 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-model-cache\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.180405 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.180333 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-home\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.180478 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.180423 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-model-cache\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.180478 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.180452 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-tmp-dir\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.182196 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.182165 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-dshm\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.182427 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.182411 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6afac70c-04d9-484d-a228-7f08d87bea15-tls-certs\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.190860 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.190838 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpjk6\" (UniqueName: \"kubernetes.io/projected/6afac70c-04d9-484d-a228-7f08d87bea15-kube-api-access-tpjk6\") pod \"rate-test-model-257300-kserve-7f887b5767-zpk2c\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.254638 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.254609 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:00.388722 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.388691 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c"] Jul 15 19:36:00.389465 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:36:00.389440 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6afac70c_04d9_484d_a228_7f08d87bea15.slice/crio-4b33c58974ed5c3c05176b502a9824c83d1581a07e0336add8516fc85c805030 WatchSource:0}: Error finding container 4b33c58974ed5c3c05176b502a9824c83d1581a07e0336add8516fc85c805030: Status 404 returned error can't find the container with id 4b33c58974ed5c3c05176b502a9824c83d1581a07e0336add8516fc85c805030 Jul 15 19:36:00.615208 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.615112 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" event={"ID":"6afac70c-04d9-484d-a228-7f08d87bea15","Type":"ContainerStarted","Data":"2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b"} Jul 15 19:36:00.615208 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.615156 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" event={"ID":"6afac70c-04d9-484d-a228-7f08d87bea15","Type":"ContainerStarted","Data":"4b33c58974ed5c3c05176b502a9824c83d1581a07e0336add8516fc85c805030"} Jul 15 19:36:00.638189 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:00.638133 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" podStartSLOduration=1.638115148 podStartE2EDuration="1.638115148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:35:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:36:00.637104661 +0000 UTC m=+2710.984816960" watchObservedRunningTime="2026-07-15 19:36:00.638115148 +0000 UTC m=+2710.985827464" Jul 15 19:36:07.391632 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.391580 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" podUID="2d76430e-eb8e-4686-a940-31bdb6be4eb0" containerName="main" containerID="cri-o://bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483" gracePeriod=45 Jul 15 19:36:07.642528 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.642471 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:36:07.650699 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.650673 2574 generic.go:356] "Generic (PLEG): container finished" podID="2d76430e-eb8e-4686-a940-31bdb6be4eb0" containerID="bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483" exitCode=0 Jul 15 19:36:07.650832 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.650702 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" Jul 15 19:36:07.650832 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.650748 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" event={"ID":"2d76430e-eb8e-4686-a940-31bdb6be4eb0","Type":"ContainerDied","Data":"bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483"} Jul 15 19:36:07.650832 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.650804 2574 scope.go:117] "RemoveContainer" containerID="bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483" Jul 15 19:36:07.650832 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.650806 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd" event={"ID":"2d76430e-eb8e-4686-a940-31bdb6be4eb0","Type":"ContainerDied","Data":"160756c6ec7719e705be4fd881427fdf8e314a4e3fc8b5c8107ab11a04b5d2da"} Jul 15 19:36:07.661014 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.660943 2574 scope.go:117] "RemoveContainer" containerID="bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483" Jul 15 19:36:07.661858 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:36:07.661830 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483\": container with ID starting with bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483 not found: ID does not exist" containerID="bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483" Jul 15 19:36:07.662028 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.661871 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483"} err="failed to get container status \"bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483\": rpc error: code = NotFound desc = could not find container \"bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483\": container with ID starting with bdc81103b5304e1e5600db198b63a7d826c1228909106d47f63b1e280ee0f483 not found: ID does not exist" Jul 15 19:36:07.746804 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.746756 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-model-cache\") pod \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " Jul 15 19:36:07.746941 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.746818 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tls-certs\") pod \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " Jul 15 19:36:07.746941 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.746864 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tmp-dir\") pod \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " Jul 15 19:36:07.746941 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.746914 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-home\") pod \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " Jul 15 19:36:07.747087 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.746958 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwgjv\" (UniqueName: \"kubernetes.io/projected/2d76430e-eb8e-4686-a940-31bdb6be4eb0-kube-api-access-pwgjv\") pod \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " Jul 15 19:36:07.747087 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.746984 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-dshm\") pod \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\" (UID: \"2d76430e-eb8e-4686-a940-31bdb6be4eb0\") " Jul 15 19:36:07.747087 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.747017 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-model-cache" (OuterVolumeSpecName: "model-cache") pod "2d76430e-eb8e-4686-a940-31bdb6be4eb0" (UID: "2d76430e-eb8e-4686-a940-31bdb6be4eb0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:36:07.747243 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.747166 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-home" (OuterVolumeSpecName: "home") pod "2d76430e-eb8e-4686-a940-31bdb6be4eb0" (UID: "2d76430e-eb8e-4686-a940-31bdb6be4eb0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:36:07.747243 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.747187 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2d76430e-eb8e-4686-a940-31bdb6be4eb0" (UID: "2d76430e-eb8e-4686-a940-31bdb6be4eb0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:36:07.747243 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.747237 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-model-cache\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:07.747375 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.747254 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tmp-dir\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:07.747375 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.747265 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-home\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:07.748983 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.748957 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2d76430e-eb8e-4686-a940-31bdb6be4eb0" (UID: "2d76430e-eb8e-4686-a940-31bdb6be4eb0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:36:07.749077 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.748993 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-dshm" (OuterVolumeSpecName: "dshm") pod "2d76430e-eb8e-4686-a940-31bdb6be4eb0" (UID: "2d76430e-eb8e-4686-a940-31bdb6be4eb0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:36:07.749077 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.749008 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d76430e-eb8e-4686-a940-31bdb6be4eb0-kube-api-access-pwgjv" (OuterVolumeSpecName: "kube-api-access-pwgjv") pod "2d76430e-eb8e-4686-a940-31bdb6be4eb0" (UID: "2d76430e-eb8e-4686-a940-31bdb6be4eb0"). InnerVolumeSpecName "kube-api-access-pwgjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:36:07.854064 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.848610 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d76430e-eb8e-4686-a940-31bdb6be4eb0-tls-certs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:07.854064 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.848642 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwgjv\" (UniqueName: \"kubernetes.io/projected/2d76430e-eb8e-4686-a940-31bdb6be4eb0-kube-api-access-pwgjv\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:07.854064 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.848658 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d76430e-eb8e-4686-a940-31bdb6be4eb0-dshm\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:07.972520 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.972493 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd"] Jul 15 19:36:07.979224 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:07.979201 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-257300/sub-test-model-257300-kserve-6f75f9cc77-dn5dd"] Jul 15 19:36:08.206274 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.206238 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d76430e-eb8e-4686-a940-31bdb6be4eb0" path="/var/lib/kubelet/pods/2d76430e-eb8e-4686-a940-31bdb6be4eb0/volumes" Jul 15 19:36:08.270012 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.269958 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" podUID="8e3ea0ef-53b9-4687-a94c-378589d5c8bf" containerName="main" containerID="cri-o://e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668" gracePeriod=45 Jul 15 19:36:08.519847 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.519824 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:36:08.655260 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.655172 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tls-certs\") pod \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " Jul 15 19:36:08.655260 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.655220 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-home\") pod \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " Jul 15 19:36:08.655489 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.655265 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-dshm\") pod \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " Jul 15 19:36:08.655489 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.655343 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-model-cache\") pod \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " Jul 15 19:36:08.655489 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.655389 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrh9d\" (UniqueName: \"kubernetes.io/projected/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-kube-api-access-jrh9d\") pod \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " Jul 15 19:36:08.655489 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.655416 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tmp-dir\") pod \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\" (UID: \"8e3ea0ef-53b9-4687-a94c-378589d5c8bf\") " Jul 15 19:36:08.655676 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.655532 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-home" (OuterVolumeSpecName: "home") pod "8e3ea0ef-53b9-4687-a94c-378589d5c8bf" (UID: "8e3ea0ef-53b9-4687-a94c-378589d5c8bf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:36:08.655676 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.655635 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-model-cache" (OuterVolumeSpecName: "model-cache") pod "8e3ea0ef-53b9-4687-a94c-378589d5c8bf" (UID: "8e3ea0ef-53b9-4687-a94c-378589d5c8bf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:36:08.655767 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.655737 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8e3ea0ef-53b9-4687-a94c-378589d5c8bf" (UID: "8e3ea0ef-53b9-4687-a94c-378589d5c8bf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:36:08.655844 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.655811 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-model-cache\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:08.655844 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.655833 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-home\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:08.657526 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.657499 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-dshm" (OuterVolumeSpecName: "dshm") pod "8e3ea0ef-53b9-4687-a94c-378589d5c8bf" (UID: "8e3ea0ef-53b9-4687-a94c-378589d5c8bf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:36:08.657526 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.657515 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-kube-api-access-jrh9d" (OuterVolumeSpecName: "kube-api-access-jrh9d") pod "8e3ea0ef-53b9-4687-a94c-378589d5c8bf" (UID: "8e3ea0ef-53b9-4687-a94c-378589d5c8bf"). InnerVolumeSpecName "kube-api-access-jrh9d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:36:08.657684 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.657635 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8e3ea0ef-53b9-4687-a94c-378589d5c8bf" (UID: "8e3ea0ef-53b9-4687-a94c-378589d5c8bf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:36:08.659436 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.659411 2574 generic.go:356] "Generic (PLEG): container finished" podID="8e3ea0ef-53b9-4687-a94c-378589d5c8bf" containerID="e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668" exitCode=0 Jul 15 19:36:08.659524 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.659453 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" Jul 15 19:36:08.659524 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.659469 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" event={"ID":"8e3ea0ef-53b9-4687-a94c-378589d5c8bf","Type":"ContainerDied","Data":"e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668"} Jul 15 19:36:08.659524 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.659509 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l" event={"ID":"8e3ea0ef-53b9-4687-a94c-378589d5c8bf","Type":"ContainerDied","Data":"318f78027ed6d3e44aa0757a3f3ededd0e31c31bbd136c2db0de41d4c06db217"} Jul 15 19:36:08.659622 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.659531 2574 scope.go:117] "RemoveContainer" containerID="e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668" Jul 15 19:36:08.675765 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.675739 2574 scope.go:117] "RemoveContainer" containerID="e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668" Jul 15 19:36:08.676016 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:36:08.675996 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668\": container with ID starting with e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668 not found: ID does not exist" containerID="e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668" Jul 15 19:36:08.676079 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.676023 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668"} err="failed to get container status \"e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668\": rpc error: code = NotFound desc = could not find container \"e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668\": container with ID starting with e1d3a10c56b03e67f2b17ad7898066fcc8bc234d0e4933b40096b110e405d668 not found: ID does not exist" Jul 15 19:36:08.692655 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.692629 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l"] Jul 15 19:36:08.699071 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.699050 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/sub-test-model-0dc297-kserve-669cc55c5c-gwv2l"] Jul 15 19:36:08.756828 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.756778 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrh9d\" (UniqueName: \"kubernetes.io/projected/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-kube-api-access-jrh9d\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:08.756828 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.756824 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tmp-dir\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:08.756972 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.756834 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-tls-certs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:08.756972 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:08.756843 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e3ea0ef-53b9-4687-a94c-378589d5c8bf-dshm\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:36:10.206373 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:10.206339 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e3ea0ef-53b9-4687-a94c-378589d5c8bf" path="/var/lib/kubelet/pods/8e3ea0ef-53b9-4687-a94c-378589d5c8bf/volumes" Jul 15 19:36:10.254824 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:10.254777 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:10.254962 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:10.254878 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:10.267851 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:10.267828 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:10.681801 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:10.681754 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:36:30.234971 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.234932 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq"] Jul 15 19:36:30.235458 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.235439 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d76430e-eb8e-4686-a940-31bdb6be4eb0" containerName="main" Jul 15 19:36:30.235504 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.235463 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d76430e-eb8e-4686-a940-31bdb6be4eb0" containerName="main" Jul 15 19:36:30.235504 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.235497 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3ea0ef-53b9-4687-a94c-378589d5c8bf" containerName="main" Jul 15 19:36:30.235567 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.235506 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3ea0ef-53b9-4687-a94c-378589d5c8bf" containerName="main" Jul 15 19:36:30.235632 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.235620 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d76430e-eb8e-4686-a940-31bdb6be4eb0" containerName="main" Jul 15 19:36:30.235684 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.235640 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e3ea0ef-53b9-4687-a94c-378589d5c8bf" containerName="main" Jul 15 19:36:30.238996 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.238974 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.241857 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.241833 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"openshift-service-ca.crt\"" Jul 15 19:36:30.242417 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.242401 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"kube-root-ca.crt\"" Jul 15 19:36:30.242496 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.242468 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"default-dockercfg-2g9wq\"" Jul 15 19:36:30.242568 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.242545 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0dc297\"/\"rate-test-model-0dc297-kserve-self-signed-certs\"" Jul 15 19:36:30.248180 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.248160 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq"] Jul 15 19:36:30.339266 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.339234 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8mhc\" (UniqueName: \"kubernetes.io/projected/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-kube-api-access-j8mhc\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.339266 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.339267 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-model-cache\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.339480 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.339340 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tmp-dir\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.339480 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.339380 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-home\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.339480 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.339413 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-dshm\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.339599 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.339482 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tls-certs\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.440032 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.440004 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tmp-dir\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.440195 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.440038 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-home\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.440195 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.440065 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-dshm\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.440195 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.440099 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tls-certs\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.440195 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.440121 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8mhc\" (UniqueName: \"kubernetes.io/projected/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-kube-api-access-j8mhc\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.440195 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.440140 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-model-cache\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.440509 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.440478 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tmp-dir\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.440638 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.440498 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-home\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.440638 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.440564 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-model-cache\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.442317 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.442300 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-dshm\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.442608 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.442589 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tls-certs\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.448612 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.448588 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8mhc\" (UniqueName: \"kubernetes.io/projected/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-kube-api-access-j8mhc\") pod \"rate-test-model-0dc297-kserve-85bb5bf959-7s2kq\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.549956 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.549932 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:30.685209 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.685181 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq"] Jul 15 19:36:30.685764 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:36:30.685737 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c292703_89d7_4e0f_a2b2_a46f65cc2cf9.slice/crio-0ac029856ee78b261127bb7c35ce3b5bc8b0bc455e54d925faca5e616a1585ae WatchSource:0}: Error finding container 0ac029856ee78b261127bb7c35ce3b5bc8b0bc455e54d925faca5e616a1585ae: Status 404 returned error can't find the container with id 0ac029856ee78b261127bb7c35ce3b5bc8b0bc455e54d925faca5e616a1585ae Jul 15 19:36:30.778020 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.777979 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" event={"ID":"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9","Type":"ContainerStarted","Data":"f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19"} Jul 15 19:36:30.778020 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.778019 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" event={"ID":"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9","Type":"ContainerStarted","Data":"0ac029856ee78b261127bb7c35ce3b5bc8b0bc455e54d925faca5e616a1585ae"} Jul 15 19:36:30.796050 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:30.795998 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" podStartSLOduration=0.795983024 podStartE2EDuration="795.983024ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:36:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:36:30.794067039 +0000 UTC m=+2741.141779331" watchObservedRunningTime="2026-07-15 19:36:30.795983024 +0000 UTC m=+2741.143695322" Jul 15 19:36:31.591468 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:31.591437 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:36:39.625698 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:39.625654 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:36:39.738021 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:39.737990 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:36:40.550493 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:40.550456 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:40.550689 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:40.550517 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:40.563053 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:40.563027 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:40.841330 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:40.841230 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:36:47.438639 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:47.438599 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:36:47.535049 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:36:47.535004 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:37:19.271663 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:19.271584 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c"] Jul 15 19:37:19.291434 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:19.291404 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:37:20.142657 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:20.142628 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq"] Jul 15 19:37:20.163428 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:20.163399 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:37:21.181467 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:21.181435 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7236-139910164246576"] Jul 15 19:37:21.184859 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:21.184843 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164246576" Jul 15 19:37:21.187459 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:21.187440 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8jrm4\"" Jul 15 19:37:21.192601 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:21.192581 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164246576"] Jul 15 19:37:21.279930 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:21.279897 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4jjs\" (UniqueName: \"kubernetes.io/projected/e9d0903a-1595-4e83-973e-7fb39fb17b7b-kube-api-access-w4jjs\") pod \"test-curl-7236-139910164246576\" (UID: \"e9d0903a-1595-4e83-973e-7fb39fb17b7b\") " pod="opendatahub/test-curl-7236-139910164246576" Jul 15 19:37:21.380347 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:21.380317 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4jjs\" (UniqueName: \"kubernetes.io/projected/e9d0903a-1595-4e83-973e-7fb39fb17b7b-kube-api-access-w4jjs\") pod \"test-curl-7236-139910164246576\" (UID: \"e9d0903a-1595-4e83-973e-7fb39fb17b7b\") " pod="opendatahub/test-curl-7236-139910164246576" Jul 15 19:37:21.391104 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:21.391081 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4jjs\" (UniqueName: \"kubernetes.io/projected/e9d0903a-1595-4e83-973e-7fb39fb17b7b-kube-api-access-w4jjs\") pod \"test-curl-7236-139910164246576\" (UID: \"e9d0903a-1595-4e83-973e-7fb39fb17b7b\") " pod="opendatahub/test-curl-7236-139910164246576" Jul 15 19:37:21.494868 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:21.494773 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164246576" Jul 15 19:37:21.626684 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:21.626659 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164246576"] Jul 15 19:37:21.628703 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:37:21.628678 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9d0903a_1595_4e83_973e_7fb39fb17b7b.slice/crio-69745792c6a39d121322ea37d74d4eff9fe449e4b75230cd889157cbdff4f0c7 WatchSource:0}: Error finding container 69745792c6a39d121322ea37d74d4eff9fe449e4b75230cd889157cbdff4f0c7: Status 404 returned error can't find the container with id 69745792c6a39d121322ea37d74d4eff9fe449e4b75230cd889157cbdff4f0c7 Jul 15 19:37:21.630352 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:21.630334 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:37:22.043918 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:22.043880 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164246576" event={"ID":"e9d0903a-1595-4e83-973e-7fb39fb17b7b","Type":"ContainerStarted","Data":"69745792c6a39d121322ea37d74d4eff9fe449e4b75230cd889157cbdff4f0c7"} Jul 15 19:37:23.050625 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:23.050582 2574 generic.go:356] "Generic (PLEG): container finished" podID="e9d0903a-1595-4e83-973e-7fb39fb17b7b" containerID="d72397b3cc79de9e2471830832a3cab62300310aa367465b2647436dc00df9ea" exitCode=0 Jul 15 19:37:23.051161 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:23.050672 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164246576" event={"ID":"e9d0903a-1595-4e83-973e-7fb39fb17b7b","Type":"ContainerDied","Data":"d72397b3cc79de9e2471830832a3cab62300310aa367465b2647436dc00df9ea"} Jul 15 19:37:24.190312 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.190286 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164246576" Jul 15 19:37:24.207380 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.207340 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4jjs\" (UniqueName: \"kubernetes.io/projected/e9d0903a-1595-4e83-973e-7fb39fb17b7b-kube-api-access-w4jjs\") pod \"e9d0903a-1595-4e83-973e-7fb39fb17b7b\" (UID: \"e9d0903a-1595-4e83-973e-7fb39fb17b7b\") " Jul 15 19:37:24.209460 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.209430 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9d0903a-1595-4e83-973e-7fb39fb17b7b-kube-api-access-w4jjs" (OuterVolumeSpecName: "kube-api-access-w4jjs") pod "e9d0903a-1595-4e83-973e-7fb39fb17b7b" (UID: "e9d0903a-1595-4e83-973e-7fb39fb17b7b"). InnerVolumeSpecName "kube-api-access-w4jjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:37:24.211869 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.211850 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7236-139910164246576_e9d0903a-1595-4e83-973e-7fb39fb17b7b/test-curl-7236-139910164246576/0.log" Jul 15 19:37:24.238364 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.238290 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7236-139910164246576"] Jul 15 19:37:24.240769 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.240751 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7236-139910164246576"] Jul 15 19:37:24.308262 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.308231 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4jjs\" (UniqueName: \"kubernetes.io/projected/e9d0903a-1595-4e83-973e-7fb39fb17b7b-kube-api-access-w4jjs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:24.379954 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.379924 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7236-139910164249904"] Jul 15 19:37:24.380267 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.380255 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9d0903a-1595-4e83-973e-7fb39fb17b7b" containerName="test-curl-7236-139910164246576" Jul 15 19:37:24.380321 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.380268 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9d0903a-1595-4e83-973e-7fb39fb17b7b" containerName="test-curl-7236-139910164246576" Jul 15 19:37:24.380363 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.380332 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9d0903a-1595-4e83-973e-7fb39fb17b7b" containerName="test-curl-7236-139910164246576" Jul 15 19:37:24.383236 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.383221 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164249904" Jul 15 19:37:24.393164 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.393142 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164249904"] Jul 15 19:37:24.408973 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.408942 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k46zp\" (UniqueName: \"kubernetes.io/projected/5a1805d0-d51b-4d43-b3d8-59154fa399df-kube-api-access-k46zp\") pod \"test-curl-7236-139910164249904\" (UID: \"5a1805d0-d51b-4d43-b3d8-59154fa399df\") " pod="opendatahub/test-curl-7236-139910164249904" Jul 15 19:37:24.509962 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.509932 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k46zp\" (UniqueName: \"kubernetes.io/projected/5a1805d0-d51b-4d43-b3d8-59154fa399df-kube-api-access-k46zp\") pod \"test-curl-7236-139910164249904\" (UID: \"5a1805d0-d51b-4d43-b3d8-59154fa399df\") " pod="opendatahub/test-curl-7236-139910164249904" Jul 15 19:37:24.518994 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.518966 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k46zp\" (UniqueName: \"kubernetes.io/projected/5a1805d0-d51b-4d43-b3d8-59154fa399df-kube-api-access-k46zp\") pod \"test-curl-7236-139910164249904\" (UID: \"5a1805d0-d51b-4d43-b3d8-59154fa399df\") " pod="opendatahub/test-curl-7236-139910164249904" Jul 15 19:37:24.693338 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.693258 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164249904" Jul 15 19:37:24.821344 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:24.821313 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164249904"] Jul 15 19:37:24.822284 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:37:24.822261 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a1805d0_d51b_4d43_b3d8_59154fa399df.slice/crio-0b5fea06762087dcbbd65b5e1d005f6d492be934af5aef26f9b47ea87872d07a WatchSource:0}: Error finding container 0b5fea06762087dcbbd65b5e1d005f6d492be934af5aef26f9b47ea87872d07a: Status 404 returned error can't find the container with id 0b5fea06762087dcbbd65b5e1d005f6d492be934af5aef26f9b47ea87872d07a Jul 15 19:37:25.068527 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:25.068497 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164246576" Jul 15 19:37:25.068695 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:25.068509 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69745792c6a39d121322ea37d74d4eff9fe449e4b75230cd889157cbdff4f0c7" Jul 15 19:37:25.069615 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:25.069590 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164249904" event={"ID":"5a1805d0-d51b-4d43-b3d8-59154fa399df","Type":"ContainerStarted","Data":"0b5fea06762087dcbbd65b5e1d005f6d492be934af5aef26f9b47ea87872d07a"} Jul 15 19:37:25.079136 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:37:25.079111 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7236-139910164246576\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" podUID="e9d0903a-1595-4e83-973e-7fb39fb17b7b" pod="opendatahub/test-curl-7236-139910164246576" Jul 15 19:37:26.075988 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:26.075954 2574 generic.go:356] "Generic (PLEG): container finished" podID="5a1805d0-d51b-4d43-b3d8-59154fa399df" containerID="10dec423f605b89a797f63822fc290b2ba4ef06a8346b986c90acf1881f7fd40" exitCode=0 Jul 15 19:37:26.075988 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:26.075992 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164249904" event={"ID":"5a1805d0-d51b-4d43-b3d8-59154fa399df","Type":"ContainerDied","Data":"10dec423f605b89a797f63822fc290b2ba4ef06a8346b986c90acf1881f7fd40"} Jul 15 19:37:26.079124 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:37:26.079086 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7236-139910164246576\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" podUID="e9d0903a-1595-4e83-973e-7fb39fb17b7b" pod="opendatahub/test-curl-7236-139910164246576" Jul 15 19:37:26.206199 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:26.206165 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9d0903a-1595-4e83-973e-7fb39fb17b7b" path="/var/lib/kubelet/pods/e9d0903a-1595-4e83-973e-7fb39fb17b7b/volumes" Jul 15 19:37:27.213026 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.213004 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164249904" Jul 15 19:37:27.234626 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.234600 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k46zp\" (UniqueName: \"kubernetes.io/projected/5a1805d0-d51b-4d43-b3d8-59154fa399df-kube-api-access-k46zp\") pod \"5a1805d0-d51b-4d43-b3d8-59154fa399df\" (UID: \"5a1805d0-d51b-4d43-b3d8-59154fa399df\") " Jul 15 19:37:27.236653 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.236625 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a1805d0-d51b-4d43-b3d8-59154fa399df-kube-api-access-k46zp" (OuterVolumeSpecName: "kube-api-access-k46zp") pod "5a1805d0-d51b-4d43-b3d8-59154fa399df" (UID: "5a1805d0-d51b-4d43-b3d8-59154fa399df"). InnerVolumeSpecName "kube-api-access-k46zp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:37:27.247855 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.247834 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7236-139910164249904_5a1805d0-d51b-4d43-b3d8-59154fa399df/test-curl-7236-139910164249904/0.log" Jul 15 19:37:27.276855 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.276829 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7236-139910164249904"] Jul 15 19:37:27.281775 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.281756 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7236-139910164249904"] Jul 15 19:37:27.336119 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.336058 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k46zp\" (UniqueName: \"kubernetes.io/projected/5a1805d0-d51b-4d43-b3d8-59154fa399df-kube-api-access-k46zp\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:27.463686 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.463652 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7236-139910164226096"] Jul 15 19:37:27.464037 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.464021 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a1805d0-d51b-4d43-b3d8-59154fa399df" containerName="test-curl-7236-139910164249904" Jul 15 19:37:27.464098 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.464040 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a1805d0-d51b-4d43-b3d8-59154fa399df" containerName="test-curl-7236-139910164249904" Jul 15 19:37:27.464139 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.464107 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a1805d0-d51b-4d43-b3d8-59154fa399df" containerName="test-curl-7236-139910164249904" Jul 15 19:37:27.467009 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.466995 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:27.473899 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.473871 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164226096"] Jul 15 19:37:27.537657 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.537633 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npv8g\" (UniqueName: \"kubernetes.io/projected/b9529bd4-148a-4ef5-baa9-44c6ff002b6c-kube-api-access-npv8g\") pod \"test-curl-7236-139910164226096\" (UID: \"b9529bd4-148a-4ef5-baa9-44c6ff002b6c\") " pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:27.638587 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.638517 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npv8g\" (UniqueName: \"kubernetes.io/projected/b9529bd4-148a-4ef5-baa9-44c6ff002b6c-kube-api-access-npv8g\") pod \"test-curl-7236-139910164226096\" (UID: \"b9529bd4-148a-4ef5-baa9-44c6ff002b6c\") " pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:27.648586 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.648562 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npv8g\" (UniqueName: \"kubernetes.io/projected/b9529bd4-148a-4ef5-baa9-44c6ff002b6c-kube-api-access-npv8g\") pod \"test-curl-7236-139910164226096\" (UID: \"b9529bd4-148a-4ef5-baa9-44c6ff002b6c\") " pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:27.776302 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.776274 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:27.903887 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:27.903860 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164226096"] Jul 15 19:37:27.905304 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:37:27.905281 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9529bd4_148a_4ef5_baa9_44c6ff002b6c.slice/crio-1ee96d54c05c0757678f1355279c2869fe1aa45093b7c9a58bc09f9fec1a18b7 WatchSource:0}: Error finding container 1ee96d54c05c0757678f1355279c2869fe1aa45093b7c9a58bc09f9fec1a18b7: Status 404 returned error can't find the container with id 1ee96d54c05c0757678f1355279c2869fe1aa45093b7c9a58bc09f9fec1a18b7 Jul 15 19:37:28.087278 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:28.087231 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164226096" event={"ID":"b9529bd4-148a-4ef5-baa9-44c6ff002b6c","Type":"ContainerStarted","Data":"1ee96d54c05c0757678f1355279c2869fe1aa45093b7c9a58bc09f9fec1a18b7"} Jul 15 19:37:28.088500 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:28.088479 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b5fea06762087dcbbd65b5e1d005f6d492be934af5aef26f9b47ea87872d07a" Jul 15 19:37:28.088500 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:28.088487 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164249904" Jul 15 19:37:28.099025 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:37:28.099001 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7236-139910164249904\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" podUID="5a1805d0-d51b-4d43-b3d8-59154fa399df" pod="opendatahub/test-curl-7236-139910164249904" Jul 15 19:37:28.206357 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:28.206297 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a1805d0-d51b-4d43-b3d8-59154fa399df" path="/var/lib/kubelet/pods/5a1805d0-d51b-4d43-b3d8-59154fa399df/volumes" Jul 15 19:37:29.094534 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:29.094506 2574 generic.go:356] "Generic (PLEG): container finished" podID="b9529bd4-148a-4ef5-baa9-44c6ff002b6c" containerID="7a11593e2efb0ab2f4ce8d7120624aa63c570a0de93f0a78f37f173ff2b80330" exitCode=0 Jul 15 19:37:29.094970 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:29.094598 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164226096" event={"ID":"b9529bd4-148a-4ef5-baa9-44c6ff002b6c","Type":"ContainerDied","Data":"7a11593e2efb0ab2f4ce8d7120624aa63c570a0de93f0a78f37f173ff2b80330"} Jul 15 19:37:29.574020 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:29.573991 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:37:29.674852 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:29.674818 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:37:30.235528 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.235506 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:30.256922 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.256896 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7236-139910164226096_b9529bd4-148a-4ef5-baa9-44c6ff002b6c/test-curl-7236-139910164226096/0.log" Jul 15 19:37:30.262011 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.261987 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npv8g\" (UniqueName: \"kubernetes.io/projected/b9529bd4-148a-4ef5-baa9-44c6ff002b6c-kube-api-access-npv8g\") pod \"b9529bd4-148a-4ef5-baa9-44c6ff002b6c\" (UID: \"b9529bd4-148a-4ef5-baa9-44c6ff002b6c\") " Jul 15 19:37:30.264106 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.264074 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9529bd4-148a-4ef5-baa9-44c6ff002b6c-kube-api-access-npv8g" (OuterVolumeSpecName: "kube-api-access-npv8g") pod "b9529bd4-148a-4ef5-baa9-44c6ff002b6c" (UID: "b9529bd4-148a-4ef5-baa9-44c6ff002b6c"). InnerVolumeSpecName "kube-api-access-npv8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:37:30.282034 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.282009 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7236-139910164226096"] Jul 15 19:37:30.287574 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.287548 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7236-139910164226096"] Jul 15 19:37:30.362943 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.362871 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npv8g\" (UniqueName: \"kubernetes.io/projected/b9529bd4-148a-4ef5-baa9-44c6ff002b6c-kube-api-access-npv8g\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:30.409983 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.409957 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7236-139910164226096"] Jul 15 19:37:30.410300 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.410287 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9529bd4-148a-4ef5-baa9-44c6ff002b6c" containerName="test-curl-7236-139910164226096" Jul 15 19:37:30.410360 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.410302 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9529bd4-148a-4ef5-baa9-44c6ff002b6c" containerName="test-curl-7236-139910164226096" Jul 15 19:37:30.410402 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.410377 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9529bd4-148a-4ef5-baa9-44c6ff002b6c" containerName="test-curl-7236-139910164226096" Jul 15 19:37:30.413350 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.413335 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:30.420112 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.420057 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164226096"] Jul 15 19:37:30.464034 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.464010 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jsbl\" (UniqueName: \"kubernetes.io/projected/7226a0f4-66e5-4bae-88e6-263bcd7a5c22-kube-api-access-4jsbl\") pod \"test-curl-7236-139910164226096\" (UID: \"7226a0f4-66e5-4bae-88e6-263bcd7a5c22\") " pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:30.565446 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.565418 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jsbl\" (UniqueName: \"kubernetes.io/projected/7226a0f4-66e5-4bae-88e6-263bcd7a5c22-kube-api-access-4jsbl\") pod \"test-curl-7236-139910164226096\" (UID: \"7226a0f4-66e5-4bae-88e6-263bcd7a5c22\") " pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:30.574724 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.574697 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jsbl\" (UniqueName: \"kubernetes.io/projected/7226a0f4-66e5-4bae-88e6-263bcd7a5c22-kube-api-access-4jsbl\") pod \"test-curl-7236-139910164226096\" (UID: \"7226a0f4-66e5-4bae-88e6-263bcd7a5c22\") " pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:30.723091 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.723024 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:30.853200 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:30.853177 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164226096"] Jul 15 19:37:30.855054 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:37:30.855022 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7226a0f4_66e5_4bae_88e6_263bcd7a5c22.slice/crio-e1e2806a0cd8a784d7fd599d7aae96e65a95184cfe6cd0149cadd28828496643 WatchSource:0}: Error finding container e1e2806a0cd8a784d7fd599d7aae96e65a95184cfe6cd0149cadd28828496643: Status 404 returned error can't find the container with id e1e2806a0cd8a784d7fd599d7aae96e65a95184cfe6cd0149cadd28828496643 Jul 15 19:37:31.106273 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:31.106239 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164226096" event={"ID":"7226a0f4-66e5-4bae-88e6-263bcd7a5c22","Type":"ContainerStarted","Data":"e1e2806a0cd8a784d7fd599d7aae96e65a95184cfe6cd0149cadd28828496643"} Jul 15 19:37:31.107378 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:31.107356 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ee96d54c05c0757678f1355279c2869fe1aa45093b7c9a58bc09f9fec1a18b7" Jul 15 19:37:31.107500 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:31.107393 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:31.118907 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:31.118879 2574 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7236-139910164226096" oldPodUID="b9529bd4-148a-4ef5-baa9-44c6ff002b6c" podUID="7226a0f4-66e5-4bae-88e6-263bcd7a5c22" Jul 15 19:37:32.114592 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:32.114559 2574 generic.go:356] "Generic (PLEG): container finished" podID="7226a0f4-66e5-4bae-88e6-263bcd7a5c22" containerID="08d6a5fc2f34dce4156a1ee844399d8f576f5f4bf2a5545e2b4ed4ad9b50480b" exitCode=0 Jul 15 19:37:32.115029 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:32.114647 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164226096" event={"ID":"7226a0f4-66e5-4bae-88e6-263bcd7a5c22","Type":"ContainerDied","Data":"08d6a5fc2f34dce4156a1ee844399d8f576f5f4bf2a5545e2b4ed4ad9b50480b"} Jul 15 19:37:32.206083 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:32.206052 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9529bd4-148a-4ef5-baa9-44c6ff002b6c" path="/var/lib/kubelet/pods/b9529bd4-148a-4ef5-baa9-44c6ff002b6c/volumes" Jul 15 19:37:33.254481 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.254456 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:33.287000 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.286974 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jsbl\" (UniqueName: \"kubernetes.io/projected/7226a0f4-66e5-4bae-88e6-263bcd7a5c22-kube-api-access-4jsbl\") pod \"7226a0f4-66e5-4bae-88e6-263bcd7a5c22\" (UID: \"7226a0f4-66e5-4bae-88e6-263bcd7a5c22\") " Jul 15 19:37:33.287603 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.287581 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7236-139910164226096_7226a0f4-66e5-4bae-88e6-263bcd7a5c22/test-curl-7236-139910164226096/0.log" Jul 15 19:37:33.289051 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.289021 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7226a0f4-66e5-4bae-88e6-263bcd7a5c22-kube-api-access-4jsbl" (OuterVolumeSpecName: "kube-api-access-4jsbl") pod "7226a0f4-66e5-4bae-88e6-263bcd7a5c22" (UID: "7226a0f4-66e5-4bae-88e6-263bcd7a5c22"). InnerVolumeSpecName "kube-api-access-4jsbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:37:33.326484 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.326458 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7236-139910164226096"] Jul 15 19:37:33.330880 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.330857 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7236-139910164226096"] Jul 15 19:37:33.387844 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.387763 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jsbl\" (UniqueName: \"kubernetes.io/projected/7226a0f4-66e5-4bae-88e6-263bcd7a5c22-kube-api-access-4jsbl\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:33.458549 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.458523 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7236-139910164227104"] Jul 15 19:37:33.458892 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.458878 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7226a0f4-66e5-4bae-88e6-263bcd7a5c22" containerName="test-curl-7236-139910164226096" Jul 15 19:37:33.458892 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.458893 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7226a0f4-66e5-4bae-88e6-263bcd7a5c22" containerName="test-curl-7236-139910164226096" Jul 15 19:37:33.458987 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.458979 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7226a0f4-66e5-4bae-88e6-263bcd7a5c22" containerName="test-curl-7236-139910164226096" Jul 15 19:37:33.461956 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.461942 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:33.468455 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.468433 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164227104"] Jul 15 19:37:33.488841 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.488813 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22htn\" (UniqueName: \"kubernetes.io/projected/ae879bf0-c809-42de-b1e6-b4f5485fce85-kube-api-access-22htn\") pod \"test-curl-7236-139910164227104\" (UID: \"ae879bf0-c809-42de-b1e6-b4f5485fce85\") " pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:33.590221 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.590191 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22htn\" (UniqueName: \"kubernetes.io/projected/ae879bf0-c809-42de-b1e6-b4f5485fce85-kube-api-access-22htn\") pod \"test-curl-7236-139910164227104\" (UID: \"ae879bf0-c809-42de-b1e6-b4f5485fce85\") " pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:33.600453 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.600430 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22htn\" (UniqueName: \"kubernetes.io/projected/ae879bf0-c809-42de-b1e6-b4f5485fce85-kube-api-access-22htn\") pod \"test-curl-7236-139910164227104\" (UID: \"ae879bf0-c809-42de-b1e6-b4f5485fce85\") " pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:33.771901 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.771873 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:33.899672 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:33.899647 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164227104"] Jul 15 19:37:33.901507 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:37:33.901483 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae879bf0_c809_42de_b1e6_b4f5485fce85.slice/crio-6e654b0f60ba31c5e7f8bacad2f60a70baab4648e2254f40bc12e125abe0a28c WatchSource:0}: Error finding container 6e654b0f60ba31c5e7f8bacad2f60a70baab4648e2254f40bc12e125abe0a28c: Status 404 returned error can't find the container with id 6e654b0f60ba31c5e7f8bacad2f60a70baab4648e2254f40bc12e125abe0a28c Jul 15 19:37:34.126836 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.126729 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164227104" event={"ID":"ae879bf0-c809-42de-b1e6-b4f5485fce85","Type":"ContainerStarted","Data":"6e654b0f60ba31c5e7f8bacad2f60a70baab4648e2254f40bc12e125abe0a28c"} Jul 15 19:37:34.127936 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.127911 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:34.128069 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.127955 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1e2806a0cd8a784d7fd599d7aae96e65a95184cfe6cd0149cadd28828496643" Jul 15 19:37:34.137985 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:37:34.137962 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7236-139910164226096\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" podUID="7226a0f4-66e5-4bae-88e6-263bcd7a5c22" pod="opendatahub/test-curl-7236-139910164226096" Jul 15 19:37:34.208241 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.208216 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7226a0f4-66e5-4bae-88e6-263bcd7a5c22" path="/var/lib/kubelet/pods/7226a0f4-66e5-4bae-88e6-263bcd7a5c22/volumes" Jul 15 19:37:34.291523 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.291489 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" podUID="6afac70c-04d9-484d-a228-7f08d87bea15" containerName="main" containerID="cri-o://2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b" gracePeriod=45 Jul 15 19:37:34.548071 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.548049 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:37:34.598378 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598352 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-tmp-dir\") pod \"6afac70c-04d9-484d-a228-7f08d87bea15\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " Jul 15 19:37:34.598507 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598384 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-home\") pod \"6afac70c-04d9-484d-a228-7f08d87bea15\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " Jul 15 19:37:34.598507 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598424 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-dshm\") pod \"6afac70c-04d9-484d-a228-7f08d87bea15\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " Jul 15 19:37:34.598507 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598459 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-model-cache\") pod \"6afac70c-04d9-484d-a228-7f08d87bea15\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " Jul 15 19:37:34.598675 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598528 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpjk6\" (UniqueName: \"kubernetes.io/projected/6afac70c-04d9-484d-a228-7f08d87bea15-kube-api-access-tpjk6\") pod \"6afac70c-04d9-484d-a228-7f08d87bea15\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " Jul 15 19:37:34.598675 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598573 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6afac70c-04d9-484d-a228-7f08d87bea15-tls-certs\") pod \"6afac70c-04d9-484d-a228-7f08d87bea15\" (UID: \"6afac70c-04d9-484d-a228-7f08d87bea15\") " Jul 15 19:37:34.598675 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598636 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-home" (OuterVolumeSpecName: "home") pod "6afac70c-04d9-484d-a228-7f08d87bea15" (UID: "6afac70c-04d9-484d-a228-7f08d87bea15"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:34.598675 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598654 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6afac70c-04d9-484d-a228-7f08d87bea15" (UID: "6afac70c-04d9-484d-a228-7f08d87bea15"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:34.598873 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598758 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-model-cache" (OuterVolumeSpecName: "model-cache") pod "6afac70c-04d9-484d-a228-7f08d87bea15" (UID: "6afac70c-04d9-484d-a228-7f08d87bea15"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:34.598937 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598899 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-tmp-dir\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:34.598937 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598918 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-home\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:34.598937 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.598932 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-model-cache\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:34.600519 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.600499 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-dshm" (OuterVolumeSpecName: "dshm") pod "6afac70c-04d9-484d-a228-7f08d87bea15" (UID: "6afac70c-04d9-484d-a228-7f08d87bea15"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:34.600610 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.600587 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6afac70c-04d9-484d-a228-7f08d87bea15-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6afac70c-04d9-484d-a228-7f08d87bea15" (UID: "6afac70c-04d9-484d-a228-7f08d87bea15"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:37:34.600645 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.600615 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6afac70c-04d9-484d-a228-7f08d87bea15-kube-api-access-tpjk6" (OuterVolumeSpecName: "kube-api-access-tpjk6") pod "6afac70c-04d9-484d-a228-7f08d87bea15" (UID: "6afac70c-04d9-484d-a228-7f08d87bea15"). InnerVolumeSpecName "kube-api-access-tpjk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:37:34.700060 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.699992 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpjk6\" (UniqueName: \"kubernetes.io/projected/6afac70c-04d9-484d-a228-7f08d87bea15-kube-api-access-tpjk6\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:34.700060 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.700015 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6afac70c-04d9-484d-a228-7f08d87bea15-tls-certs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:34.700060 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:34.700024 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6afac70c-04d9-484d-a228-7f08d87bea15-dshm\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:35.134890 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.134854 2574 generic.go:356] "Generic (PLEG): container finished" podID="ae879bf0-c809-42de-b1e6-b4f5485fce85" containerID="49a3f891c23d492acecb023369d15adb76a324683f750bf94070162d9dc5d651" exitCode=0 Jul 15 19:37:35.135045 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.134946 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164227104" event={"ID":"ae879bf0-c809-42de-b1e6-b4f5485fce85","Type":"ContainerDied","Data":"49a3f891c23d492acecb023369d15adb76a324683f750bf94070162d9dc5d651"} Jul 15 19:37:35.136130 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.136103 2574 generic.go:356] "Generic (PLEG): container finished" podID="6afac70c-04d9-484d-a228-7f08d87bea15" containerID="2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b" exitCode=0 Jul 15 19:37:35.136259 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.136138 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" Jul 15 19:37:35.136259 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.136142 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" event={"ID":"6afac70c-04d9-484d-a228-7f08d87bea15","Type":"ContainerDied","Data":"2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b"} Jul 15 19:37:35.136259 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.136179 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c" event={"ID":"6afac70c-04d9-484d-a228-7f08d87bea15","Type":"ContainerDied","Data":"4b33c58974ed5c3c05176b502a9824c83d1581a07e0336add8516fc85c805030"} Jul 15 19:37:35.136259 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.136197 2574 scope.go:117] "RemoveContainer" containerID="2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b" Jul 15 19:37:35.146117 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.146100 2574 scope.go:117] "RemoveContainer" containerID="2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b" Jul 15 19:37:35.146395 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:37:35.146375 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b\": container with ID starting with 2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b not found: ID does not exist" containerID="2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b" Jul 15 19:37:35.146477 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.146402 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b"} err="failed to get container status \"2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b\": rpc error: code = NotFound desc = could not find container \"2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b\": container with ID starting with 2d50fdd54b26393a3428221c3a8bc062513c3cd49eb4f0b8f781f1d71a13132b not found: ID does not exist" Jul 15 19:37:35.161403 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.161383 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c"] Jul 15 19:37:35.161963 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.161944 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" podUID="7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" containerName="main" containerID="cri-o://f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19" gracePeriod=45 Jul 15 19:37:35.165808 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.165772 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-257300/rate-test-model-257300-kserve-7f887b5767-zpk2c"] Jul 15 19:37:35.416797 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.416760 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:37:35.507296 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.507268 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8mhc\" (UniqueName: \"kubernetes.io/projected/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-kube-api-access-j8mhc\") pod \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " Jul 15 19:37:35.507464 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.507303 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-home\") pod \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " Jul 15 19:37:35.507464 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.507345 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-dshm\") pod \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " Jul 15 19:37:35.507464 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.507385 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tls-certs\") pod \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " Jul 15 19:37:35.507464 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.507413 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tmp-dir\") pod \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " Jul 15 19:37:35.507464 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.507444 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-model-cache\") pod \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\" (UID: \"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9\") " Jul 15 19:37:35.507759 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.507576 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-home" (OuterVolumeSpecName: "home") pod "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" (UID: "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:35.507759 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.507683 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" (UID: "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:35.507759 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.507711 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-home\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:35.507759 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.507708 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-model-cache" (OuterVolumeSpecName: "model-cache") pod "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" (UID: "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:35.509465 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.509443 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-kube-api-access-j8mhc" (OuterVolumeSpecName: "kube-api-access-j8mhc") pod "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" (UID: "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9"). InnerVolumeSpecName "kube-api-access-j8mhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:37:35.509922 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.509896 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-dshm" (OuterVolumeSpecName: "dshm") pod "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" (UID: "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:35.510005 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.509915 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" (UID: "7c292703-89d7-4e0f-a2b2-a46f65cc2cf9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:37:35.608399 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.608371 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tmp-dir\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:35.608399 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.608393 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-model-cache\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:35.608399 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.608404 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8mhc\" (UniqueName: \"kubernetes.io/projected/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-kube-api-access-j8mhc\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:35.608585 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.608413 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-dshm\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:35.608585 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:35.608421 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9-tls-certs\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:36.142831 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.142795 2574 generic.go:356] "Generic (PLEG): container finished" podID="7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" containerID="f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19" exitCode=0 Jul 15 19:37:36.143018 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.142841 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" Jul 15 19:37:36.143018 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.142880 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" event={"ID":"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9","Type":"ContainerDied","Data":"f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19"} Jul 15 19:37:36.143018 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.142923 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq" event={"ID":"7c292703-89d7-4e0f-a2b2-a46f65cc2cf9","Type":"ContainerDied","Data":"0ac029856ee78b261127bb7c35ce3b5bc8b0bc455e54d925faca5e616a1585ae"} Jul 15 19:37:36.143018 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.142942 2574 scope.go:117] "RemoveContainer" containerID="f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19" Jul 15 19:37:36.152392 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.152370 2574 scope.go:117] "RemoveContainer" containerID="f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19" Jul 15 19:37:36.152657 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:37:36.152638 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19\": container with ID starting with f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19 not found: ID does not exist" containerID="f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19" Jul 15 19:37:36.152702 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.152666 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19"} err="failed to get container status \"f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19\": rpc error: code = NotFound desc = could not find container \"f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19\": container with ID starting with f6b83f204682cc139bcf14c48a527268220dcd6064bee3d54f008d717986db19 not found: ID does not exist" Jul 15 19:37:36.164734 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.164708 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq"] Jul 15 19:37:36.169071 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.169048 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0dc297/rate-test-model-0dc297-kserve-85bb5bf959-7s2kq"] Jul 15 19:37:36.216826 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.216769 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6afac70c-04d9-484d-a228-7f08d87bea15" path="/var/lib/kubelet/pods/6afac70c-04d9-484d-a228-7f08d87bea15/volumes" Jul 15 19:37:36.217256 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.217238 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" path="/var/lib/kubelet/pods/7c292703-89d7-4e0f-a2b2-a46f65cc2cf9/volumes" Jul 15 19:37:36.277518 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.277496 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:36.311174 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.311150 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7236-139910164227104_ae879bf0-c809-42de-b1e6-b4f5485fce85/test-curl-7236-139910164227104/0.log" Jul 15 19:37:36.315397 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.315374 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22htn\" (UniqueName: \"kubernetes.io/projected/ae879bf0-c809-42de-b1e6-b4f5485fce85-kube-api-access-22htn\") pod \"ae879bf0-c809-42de-b1e6-b4f5485fce85\" (UID: \"ae879bf0-c809-42de-b1e6-b4f5485fce85\") " Jul 15 19:37:36.317251 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.317228 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae879bf0-c809-42de-b1e6-b4f5485fce85-kube-api-access-22htn" (OuterVolumeSpecName: "kube-api-access-22htn") pod "ae879bf0-c809-42de-b1e6-b4f5485fce85" (UID: "ae879bf0-c809-42de-b1e6-b4f5485fce85"). InnerVolumeSpecName "kube-api-access-22htn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:37:36.360261 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.360236 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7236-139910164227104"] Jul 15 19:37:36.373602 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.373579 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7236-139910164227104"] Jul 15 19:37:36.416507 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.416452 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22htn\" (UniqueName: \"kubernetes.io/projected/ae879bf0-c809-42de-b1e6-b4f5485fce85-kube-api-access-22htn\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:36.500161 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.500137 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7236-139910164227104"] Jul 15 19:37:36.500513 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.500452 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6afac70c-04d9-484d-a228-7f08d87bea15" containerName="main" Jul 15 19:37:36.500513 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.500463 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6afac70c-04d9-484d-a228-7f08d87bea15" containerName="main" Jul 15 19:37:36.500513 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.500472 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae879bf0-c809-42de-b1e6-b4f5485fce85" containerName="test-curl-7236-139910164227104" Jul 15 19:37:36.500513 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.500477 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae879bf0-c809-42de-b1e6-b4f5485fce85" containerName="test-curl-7236-139910164227104" Jul 15 19:37:36.500513 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.500488 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" containerName="main" Jul 15 19:37:36.500513 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.500494 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" containerName="main" Jul 15 19:37:36.500718 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.500549 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae879bf0-c809-42de-b1e6-b4f5485fce85" containerName="test-curl-7236-139910164227104" Jul 15 19:37:36.500718 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.500558 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c292703-89d7-4e0f-a2b2-a46f65cc2cf9" containerName="main" Jul 15 19:37:36.500718 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.500564 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6afac70c-04d9-484d-a228-7f08d87bea15" containerName="main" Jul 15 19:37:36.503562 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.503547 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:36.511401 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.511377 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164227104"] Jul 15 19:37:36.618390 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.618357 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcb9s\" (UniqueName: \"kubernetes.io/projected/fca33421-28cb-42c0-a7da-10e68d3f3096-kube-api-access-zcb9s\") pod \"test-curl-7236-139910164227104\" (UID: \"fca33421-28cb-42c0-a7da-10e68d3f3096\") " pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:36.718955 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.718885 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcb9s\" (UniqueName: \"kubernetes.io/projected/fca33421-28cb-42c0-a7da-10e68d3f3096-kube-api-access-zcb9s\") pod \"test-curl-7236-139910164227104\" (UID: \"fca33421-28cb-42c0-a7da-10e68d3f3096\") " pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:36.727505 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.727482 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcb9s\" (UniqueName: \"kubernetes.io/projected/fca33421-28cb-42c0-a7da-10e68d3f3096-kube-api-access-zcb9s\") pod \"test-curl-7236-139910164227104\" (UID: \"fca33421-28cb-42c0-a7da-10e68d3f3096\") " pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:36.813387 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.813350 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:36.937142 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:36.937115 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7236-139910164227104"] Jul 15 19:37:36.939150 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:37:36.939120 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfca33421_28cb_42c0_a7da_10e68d3f3096.slice/crio-cd35c36f271f92b7f5a5b81189dfe7b3fd393f8306a36aafe0847765c45eb25f WatchSource:0}: Error finding container cd35c36f271f92b7f5a5b81189dfe7b3fd393f8306a36aafe0847765c45eb25f: Status 404 returned error can't find the container with id cd35c36f271f92b7f5a5b81189dfe7b3fd393f8306a36aafe0847765c45eb25f Jul 15 19:37:37.150286 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:37.150252 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164227104" event={"ID":"fca33421-28cb-42c0-a7da-10e68d3f3096","Type":"ContainerStarted","Data":"cd35c36f271f92b7f5a5b81189dfe7b3fd393f8306a36aafe0847765c45eb25f"} Jul 15 19:37:37.151425 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:37.151409 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e654b0f60ba31c5e7f8bacad2f60a70baab4648e2254f40bc12e125abe0a28c" Jul 15 19:37:37.151537 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:37.151448 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:37.163284 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:37.163259 2574 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7236-139910164227104" oldPodUID="ae879bf0-c809-42de-b1e6-b4f5485fce85" podUID="fca33421-28cb-42c0-a7da-10e68d3f3096" Jul 15 19:37:37.379546 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:37:37.379520 2574 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfca33421_28cb_42c0_a7da_10e68d3f3096.slice/crio-f424e12a54ac598decffbc4d8768aaf73dc670e5e5eee5bedbd3d52be061f627.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfca33421_28cb_42c0_a7da_10e68d3f3096.slice/crio-conmon-f424e12a54ac598decffbc4d8768aaf73dc670e5e5eee5bedbd3d52be061f627.scope\": RecentStats: unable to find data in memory cache]" Jul 15 19:37:38.157702 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:38.157665 2574 generic.go:356] "Generic (PLEG): container finished" podID="fca33421-28cb-42c0-a7da-10e68d3f3096" containerID="f424e12a54ac598decffbc4d8768aaf73dc670e5e5eee5bedbd3d52be061f627" exitCode=0 Jul 15 19:37:38.157702 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:38.157704 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7236-139910164227104" event={"ID":"fca33421-28cb-42c0-a7da-10e68d3f3096","Type":"ContainerDied","Data":"f424e12a54ac598decffbc4d8768aaf73dc670e5e5eee5bedbd3d52be061f627"} Jul 15 19:37:38.205449 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:38.205421 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae879bf0-c809-42de-b1e6-b4f5485fce85" path="/var/lib/kubelet/pods/ae879bf0-c809-42de-b1e6-b4f5485fce85/volumes" Jul 15 19:37:39.294893 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.294873 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:39.315396 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.315375 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7236-139910164227104_fca33421-28cb-42c0-a7da-10e68d3f3096/test-curl-7236-139910164227104/0.log" Jul 15 19:37:39.338990 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.338970 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcb9s\" (UniqueName: \"kubernetes.io/projected/fca33421-28cb-42c0-a7da-10e68d3f3096-kube-api-access-zcb9s\") pod \"fca33421-28cb-42c0-a7da-10e68d3f3096\" (UID: \"fca33421-28cb-42c0-a7da-10e68d3f3096\") " Jul 15 19:37:39.341152 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.341120 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fca33421-28cb-42c0-a7da-10e68d3f3096-kube-api-access-zcb9s" (OuterVolumeSpecName: "kube-api-access-zcb9s") pod "fca33421-28cb-42c0-a7da-10e68d3f3096" (UID: "fca33421-28cb-42c0-a7da-10e68d3f3096"). InnerVolumeSpecName "kube-api-access-zcb9s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:37:39.342599 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.342580 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7236-139910164227104"] Jul 15 19:37:39.346623 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.346598 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7236-139910164227104"] Jul 15 19:37:39.439664 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.439578 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcb9s\" (UniqueName: \"kubernetes.io/projected/fca33421-28cb-42c0-a7da-10e68d3f3096-kube-api-access-zcb9s\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:39.543392 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.543360 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp"] Jul 15 19:37:39.543687 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.543676 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fca33421-28cb-42c0-a7da-10e68d3f3096" containerName="test-curl-7236-139910164227104" Jul 15 19:37:39.543726 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.543689 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fca33421-28cb-42c0-a7da-10e68d3f3096" containerName="test-curl-7236-139910164227104" Jul 15 19:37:39.543804 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.543762 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fca33421-28cb-42c0-a7da-10e68d3f3096" containerName="test-curl-7236-139910164227104" Jul 15 19:37:39.546829 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.546812 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp" Jul 15 19:37:39.549301 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.549271 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2f4w7\"" Jul 15 19:37:39.549411 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.549335 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 19:37:39.553925 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.553865 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp"] Jul 15 19:37:39.641038 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.641007 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9520af9e-fbcf-45b8-b8ab-38c0cf315adc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-257300-mstvp\" (UID: \"9520af9e-fbcf-45b8-b8ab-38c0cf315adc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp" Jul 15 19:37:39.742480 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.742400 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9520af9e-fbcf-45b8-b8ab-38c0cf315adc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-257300-mstvp\" (UID: \"9520af9e-fbcf-45b8-b8ab-38c0cf315adc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp" Jul 15 19:37:39.743080 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.743057 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9520af9e-fbcf-45b8-b8ab-38c0cf315adc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-257300-mstvp\" (UID: \"9520af9e-fbcf-45b8-b8ab-38c0cf315adc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp" Jul 15 19:37:39.858083 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.858053 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp" Jul 15 19:37:39.984646 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:37:39.984615 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9520af9e_fbcf_45b8_b8ab_38c0cf315adc.slice/crio-e61b5556b14e2475b714e25c1da2dd429f410a5ea509c19076a784cda19f3983 WatchSource:0}: Error finding container e61b5556b14e2475b714e25c1da2dd429f410a5ea509c19076a784cda19f3983: Status 404 returned error can't find the container with id e61b5556b14e2475b714e25c1da2dd429f410a5ea509c19076a784cda19f3983 Jul 15 19:37:39.985282 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:39.985257 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp"] Jul 15 19:37:40.077931 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:40.077906 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:37:40.168987 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:40.168956 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd35c36f271f92b7f5a5b81189dfe7b3fd393f8306a36aafe0847765c45eb25f" Jul 15 19:37:40.168987 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:40.168962 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:40.170100 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:40.170068 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp" event={"ID":"9520af9e-fbcf-45b8-b8ab-38c0cf315adc","Type":"ContainerStarted","Data":"e61b5556b14e2475b714e25c1da2dd429f410a5ea509c19076a784cda19f3983"} Jul 15 19:37:40.179586 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:37:40.179559 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7236-139910164227104\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" podUID="fca33421-28cb-42c0-a7da-10e68d3f3096" pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:40.181838 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:40.181810 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:37:40.204900 ip-10-0-137-11 kubenswrapper[2574]: E0715 19:37:40.204863 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7236-139910164227104\" is forbidden: User \"system:node:ip-10-0-137-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-11.ec2.internal' and this object" podUID="fca33421-28cb-42c0-a7da-10e68d3f3096" pod="opendatahub/test-curl-7236-139910164227104" Jul 15 19:37:40.206210 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:40.206190 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fca33421-28cb-42c0-a7da-10e68d3f3096" path="/var/lib/kubelet/pods/fca33421-28cb-42c0-a7da-10e68d3f3096/volumes" Jul 15 19:37:41.176802 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:41.176753 2574 generic.go:356] "Generic (PLEG): container finished" podID="9520af9e-fbcf-45b8-b8ab-38c0cf315adc" containerID="858aee02a9c5a721ef1d99d478f001ec2de83bf9cd4a5a6c0dbc75a5da25f993" exitCode=0 Jul 15 19:37:41.177176 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:41.176836 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp" event={"ID":"9520af9e-fbcf-45b8-b8ab-38c0cf315adc","Type":"ContainerDied","Data":"858aee02a9c5a721ef1d99d478f001ec2de83bf9cd4a5a6c0dbc75a5da25f993"} Jul 15 19:37:42.313827 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:42.313802 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp" Jul 15 19:37:42.362538 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:42.362512 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9520af9e-fbcf-45b8-b8ab-38c0cf315adc-maas-api-service-ca\") pod \"9520af9e-fbcf-45b8-b8ab-38c0cf315adc\" (UID: \"9520af9e-fbcf-45b8-b8ab-38c0cf315adc\") " Jul 15 19:37:42.362879 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:42.362859 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9520af9e-fbcf-45b8-b8ab-38c0cf315adc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9520af9e-fbcf-45b8-b8ab-38c0cf315adc" (UID: "9520af9e-fbcf-45b8-b8ab-38c0cf315adc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:37:42.464209 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:42.464143 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9520af9e-fbcf-45b8-b8ab-38c0cf315adc-maas-api-service-ca\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:37:43.188171 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:43.188139 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp" Jul 15 19:37:43.188342 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:43.188186 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-257300-mstvp" event={"ID":"9520af9e-fbcf-45b8-b8ab-38c0cf315adc","Type":"ContainerDied","Data":"e61b5556b14e2475b714e25c1da2dd429f410a5ea509c19076a784cda19f3983"} Jul 15 19:37:43.188342 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:43.188228 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e61b5556b14e2475b714e25c1da2dd429f410a5ea509c19076a784cda19f3983" Jul 15 19:37:50.183890 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:50.183855 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:37:54.742034 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:37:54.741992 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:01.430260 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:01.430229 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:01.503983 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:01.503950 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:03.244705 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:03.244670 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:05.524055 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:05.524013 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:05.795746 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:05.795659 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:05.919522 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:05.919480 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv"] Jul 15 19:38:05.919880 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:05.919868 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9520af9e-fbcf-45b8-b8ab-38c0cf315adc" containerName="revoke-keys" Jul 15 19:38:05.919937 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:05.919882 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9520af9e-fbcf-45b8-b8ab-38c0cf315adc" containerName="revoke-keys" Jul 15 19:38:05.919973 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:05.919955 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9520af9e-fbcf-45b8-b8ab-38c0cf315adc" containerName="revoke-keys" Jul 15 19:38:05.923051 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:05.923034 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv" Jul 15 19:38:05.925646 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:05.925623 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2f4w7\"" Jul 15 19:38:05.925807 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:05.925627 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 19:38:05.930294 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:05.930257 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv"] Jul 15 19:38:06.058630 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:06.058535 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f30c834-14b4-4ecc-8570-709b468ff162-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv\" (UID: \"7f30c834-14b4-4ecc-8570-709b468ff162\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv" Jul 15 19:38:06.159999 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:06.159963 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f30c834-14b4-4ecc-8570-709b468ff162-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv\" (UID: \"7f30c834-14b4-4ecc-8570-709b468ff162\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv" Jul 15 19:38:06.160631 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:06.160608 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f30c834-14b4-4ecc-8570-709b468ff162-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv\" (UID: \"7f30c834-14b4-4ecc-8570-709b468ff162\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv" Jul 15 19:38:06.235780 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:06.235736 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv" Jul 15 19:38:06.377965 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:06.377842 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv"] Jul 15 19:38:06.380197 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:38:06.380159 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f30c834_14b4_4ecc_8570_709b468ff162.slice/crio-dd3dac61ed8b328bff6507c99cb0bfd0c43a91ce2053c129ea26aca602ed8b09 WatchSource:0}: Error finding container dd3dac61ed8b328bff6507c99cb0bfd0c43a91ce2053c129ea26aca602ed8b09: Status 404 returned error can't find the container with id dd3dac61ed8b328bff6507c99cb0bfd0c43a91ce2053c129ea26aca602ed8b09 Jul 15 19:38:06.464369 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:06.464327 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:07.315658 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:07.315626 2574 generic.go:356] "Generic (PLEG): container finished" podID="7f30c834-14b4-4ecc-8570-709b468ff162" containerID="471029f9fea3d8041da43b50bb0de2f17a052beba9500b63cc6641c18e8cd91b" exitCode=0 Jul 15 19:38:07.316092 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:07.315720 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv" event={"ID":"7f30c834-14b4-4ecc-8570-709b468ff162","Type":"ContainerDied","Data":"471029f9fea3d8041da43b50bb0de2f17a052beba9500b63cc6641c18e8cd91b"} Jul 15 19:38:07.316092 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:07.315808 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv" event={"ID":"7f30c834-14b4-4ecc-8570-709b468ff162","Type":"ContainerStarted","Data":"dd3dac61ed8b328bff6507c99cb0bfd0c43a91ce2053c129ea26aca602ed8b09"} Jul 15 19:38:08.402229 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:08.402192 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:08.453740 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:08.453715 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv" Jul 15 19:38:08.581562 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:08.581455 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f30c834-14b4-4ecc-8570-709b468ff162-maas-api-service-ca\") pod \"7f30c834-14b4-4ecc-8570-709b468ff162\" (UID: \"7f30c834-14b4-4ecc-8570-709b468ff162\") " Jul 15 19:38:08.581895 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:08.581874 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f30c834-14b4-4ecc-8570-709b468ff162-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f30c834-14b4-4ecc-8570-709b468ff162" (UID: "7f30c834-14b4-4ecc-8570-709b468ff162"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:38:08.682424 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:08.682381 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f30c834-14b4-4ecc-8570-709b468ff162-maas-api-service-ca\") on node \"ip-10-0-137-11.ec2.internal\" DevicePath \"\"" Jul 15 19:38:09.327845 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:09.327817 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv" Jul 15 19:38:09.328022 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:09.327850 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv" event={"ID":"7f30c834-14b4-4ecc-8570-709b468ff162","Type":"ContainerDied","Data":"dd3dac61ed8b328bff6507c99cb0bfd0c43a91ce2053c129ea26aca602ed8b09"} Jul 15 19:38:09.328022 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:09.327893 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd3dac61ed8b328bff6507c99cb0bfd0c43a91ce2053c129ea26aca602ed8b09" Jul 15 19:38:16.423742 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:16.423710 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:21.015702 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:21.015666 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:27.134210 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:27.134176 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:27.256741 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:27.256711 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:31.761800 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:31.761750 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:32.008089 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:32.008047 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:32.433333 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:32.433293 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7964cb565-h9ffr_c5c79352-4fa4-48e2-86f3-a754c08e393d/authorino/0.log" Jul 15 19:38:38.515080 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:38.515045 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7dcfcc7558-xnh2k_180331a3-7b84-4974-8c02-e91cc5148a9b/manager/0.log" Jul 15 19:38:38.643651 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:38.643626 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cfwq4_587d6702-7ab8-425d-afc1-3e60ab776316/server/0.log" Jul 15 19:38:38.773098 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:38.773015 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-d7995_d5cf65fc-bc88-4162-8f42-19760b3b9be3/manager/0.log" Jul 15 19:38:39.164371 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:39.164293 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dbd7b8848-wd9vw_12951936-1742-485e-a2a2-7d3b57a86f88/manager/0.log" Jul 15 19:38:39.181295 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:39.181274 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dbd7b8848-wd9vw_12951936-1742-485e-a2a2-7d3b57a86f88/copy-manifests/0.log" Jul 15 19:38:39.813377 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:39.813345 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735730-vpkwd_96b31560-730c-4d14-9146-b731d1edf91e/cleanup/0.log" Jul 15 19:38:40.301869 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:40.301839 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-97fde9-xgwxt_7a8702ca-9d4a-4249-a49c-086e98395ecc/revoke-keys/0.log" Jul 15 19:38:41.143000 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:41.142971 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-257300-mstvp_9520af9e-fbcf-45b8-b8ab-38c0cf315adc/revoke-keys/0.log" Jul 15 19:38:41.263410 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:41.263384 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-0dc297-v6qnv_7f30c834-14b4-4ecc-8570-709b468ff162/revoke-keys/0.log" Jul 15 19:38:42.242825 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.242773 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4_06d1daf3-2997-459d-92ff-b0932eb82af8/util/0.log" Jul 15 19:38:42.250353 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.250330 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4_06d1daf3-2997-459d-92ff-b0932eb82af8/pull/0.log" Jul 15 19:38:42.257172 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.257150 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4_06d1daf3-2997-459d-92ff-b0932eb82af8/extract/0.log" Jul 15 19:38:42.379732 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.379702 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526_60e0d4e3-2398-41f3-a06e-c3c3afaa5022/util/0.log" Jul 15 19:38:42.386777 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.386753 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526_60e0d4e3-2398-41f3-a06e-c3c3afaa5022/pull/0.log" Jul 15 19:38:42.393546 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.393527 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526_60e0d4e3-2398-41f3-a06e-c3c3afaa5022/extract/0.log" Jul 15 19:38:42.516661 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.516632 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n_9fc5d28f-47ad-4c15-a291-210c00fc1cd7/util/0.log" Jul 15 19:38:42.524392 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.524367 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n_9fc5d28f-47ad-4c15-a291-210c00fc1cd7/pull/0.log" Jul 15 19:38:42.531411 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.531384 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n_9fc5d28f-47ad-4c15-a291-210c00fc1cd7/extract/0.log" Jul 15 19:38:42.649333 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.649305 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s_77f8dc6a-3ad1-4dad-a529-a2d9616e8274/util/0.log" Jul 15 19:38:42.656036 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.656013 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s_77f8dc6a-3ad1-4dad-a529-a2d9616e8274/pull/0.log" Jul 15 19:38:42.662258 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.662239 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s_77f8dc6a-3ad1-4dad-a529-a2d9616e8274/extract/0.log" Jul 15 19:38:42.787470 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:42.787384 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7964cb565-h9ffr_c5c79352-4fa4-48e2-86f3-a754c08e393d/authorino/0.log" Jul 15 19:38:43.147987 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:43.147910 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rmv8k_adbc93c7-471f-4406-beeb-e748d9cee3e8/kuadrant-console-plugin/0.log" Jul 15 19:38:44.309136 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:44.309107 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbd849f4d-98bjb_596dd6fd-f306-4cdf-ba48-3f73028221c0/kube-auth-proxy/0.log" Jul 15 19:38:44.797228 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:44.797195 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-22jvw_0827f76e-ef0a-44c1-87b9-8ee6eef709be/payload-processing/0.log" Jul 15 19:38:45.408008 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:45.407980 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-zkb26_86963b61-152e-475c-8b1a-7352c03f6a13/main/0.log" Jul 15 19:38:45.527914 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:45.527881 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-cp7rn_7958b7e7-964b-4e55-81be-63c7546bb850/main/0.log" Jul 15 19:38:45.882393 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:45.882362 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-nkkz9_d57a2481-7549-4855-9a7d-95b5d8bc079d/main/0.log" Jul 15 19:38:52.487389 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.487350 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6vp87/must-gather-cjpc5"] Jul 15 19:38:52.487931 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.487905 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f30c834-14b4-4ecc-8570-709b468ff162" containerName="revoke-keys" Jul 15 19:38:52.487931 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.487925 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f30c834-14b4-4ecc-8570-709b468ff162" containerName="revoke-keys" Jul 15 19:38:52.488055 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.488042 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f30c834-14b4-4ecc-8570-709b468ff162" containerName="revoke-keys" Jul 15 19:38:52.491493 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.491471 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vp87/must-gather-cjpc5" Jul 15 19:38:52.494684 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.494658 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6vp87\"/\"default-dockercfg-ttwfw\"" Jul 15 19:38:52.495319 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.495302 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6vp87\"/\"kube-root-ca.crt\"" Jul 15 19:38:52.495390 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.495335 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6vp87\"/\"openshift-service-ca.crt\"" Jul 15 19:38:52.501563 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.501537 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vp87/must-gather-cjpc5"] Jul 15 19:38:52.509194 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.509162 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:52.556051 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.556005 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwwkz\" (UniqueName: \"kubernetes.io/projected/d9434d86-f824-4d50-9770-8f56f43398b2-kube-api-access-xwwkz\") pod \"must-gather-cjpc5\" (UID: \"d9434d86-f824-4d50-9770-8f56f43398b2\") " pod="openshift-must-gather-6vp87/must-gather-cjpc5" Jul 15 19:38:52.556224 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.556070 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d9434d86-f824-4d50-9770-8f56f43398b2-must-gather-output\") pod \"must-gather-cjpc5\" (UID: \"d9434d86-f824-4d50-9770-8f56f43398b2\") " pod="openshift-must-gather-6vp87/must-gather-cjpc5" Jul 15 19:38:52.600503 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.600473 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:38:52.656818 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.656758 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwwkz\" (UniqueName: \"kubernetes.io/projected/d9434d86-f824-4d50-9770-8f56f43398b2-kube-api-access-xwwkz\") pod \"must-gather-cjpc5\" (UID: \"d9434d86-f824-4d50-9770-8f56f43398b2\") " pod="openshift-must-gather-6vp87/must-gather-cjpc5" Jul 15 19:38:52.656987 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.656842 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d9434d86-f824-4d50-9770-8f56f43398b2-must-gather-output\") pod \"must-gather-cjpc5\" (UID: \"d9434d86-f824-4d50-9770-8f56f43398b2\") " pod="openshift-must-gather-6vp87/must-gather-cjpc5" Jul 15 19:38:52.657144 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.657128 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d9434d86-f824-4d50-9770-8f56f43398b2-must-gather-output\") pod \"must-gather-cjpc5\" (UID: \"d9434d86-f824-4d50-9770-8f56f43398b2\") " pod="openshift-must-gather-6vp87/must-gather-cjpc5" Jul 15 19:38:52.667645 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.667618 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwwkz\" (UniqueName: \"kubernetes.io/projected/d9434d86-f824-4d50-9770-8f56f43398b2-kube-api-access-xwwkz\") pod \"must-gather-cjpc5\" (UID: \"d9434d86-f824-4d50-9770-8f56f43398b2\") " pod="openshift-must-gather-6vp87/must-gather-cjpc5" Jul 15 19:38:52.802007 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.801964 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vp87/must-gather-cjpc5" Jul 15 19:38:52.963826 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:52.963799 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vp87/must-gather-cjpc5"] Jul 15 19:38:52.965198 ip-10-0-137-11 kubenswrapper[2574]: W0715 19:38:52.965176 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9434d86_f824_4d50_9770_8f56f43398b2.slice/crio-24ce6d63658fa39a85d5bf32505369eb1998f240562b74eea212f0442cde1e84 WatchSource:0}: Error finding container 24ce6d63658fa39a85d5bf32505369eb1998f240562b74eea212f0442cde1e84: Status 404 returned error can't find the container with id 24ce6d63658fa39a85d5bf32505369eb1998f240562b74eea212f0442cde1e84 Jul 15 19:38:53.569432 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:53.569386 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vp87/must-gather-cjpc5" event={"ID":"d9434d86-f824-4d50-9770-8f56f43398b2","Type":"ContainerStarted","Data":"24ce6d63658fa39a85d5bf32505369eb1998f240562b74eea212f0442cde1e84"} Jul 15 19:38:54.586026 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:54.585834 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vp87/must-gather-cjpc5" event={"ID":"d9434d86-f824-4d50-9770-8f56f43398b2","Type":"ContainerStarted","Data":"02c59cf8d7e21a1ee6c047a2fe9bfbbde2e0e7a7e431a3172d3cfaa4a47fcbc4"} Jul 15 19:38:54.586026 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:54.585887 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vp87/must-gather-cjpc5" event={"ID":"d9434d86-f824-4d50-9770-8f56f43398b2","Type":"ContainerStarted","Data":"26c0dfc5124edb7f18a64c67debdb4f52afc79865afd01ea38a86a3c50ef3d20"} Jul 15 19:38:54.604592 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:54.604509 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6vp87/must-gather-cjpc5" podStartSLOduration=1.718144364 podStartE2EDuration="2.60448635s" totalImagesPullingTime="886.341986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:38:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:38:54.602692293 +0000 UTC m=+2884.950404592" watchObservedRunningTime="2026-07-15 19:38:54.60448635 +0000 UTC m=+2884.952198654" Jul 15 19:38:55.649058 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:55.649021 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z65fv_ae918cfe-ebcc-4398-b50c-dacff69aa164/global-pull-secret-syncer/0.log" Jul 15 19:38:55.702486 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:55.702457 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" Jul 15 19:38:55.839500 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:55.839462 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-11.ec2.internal_6eba61179e763a6a602c3d40734955c6/haproxy/0.log" Jul 15 19:38:59.965557 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:59.965485 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4_06d1daf3-2997-459d-92ff-b0932eb82af8/extract/0.log" Jul 15 19:38:59.991079 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:38:59.990933 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4_06d1daf3-2997-459d-92ff-b0932eb82af8/util/0.log" Jul 15 19:39:00.018702 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.018651 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594f4b4_06d1daf3-2997-459d-92ff-b0932eb82af8/pull/0.log" Jul 15 19:39:00.063745 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.063711 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526_60e0d4e3-2398-41f3-a06e-c3c3afaa5022/extract/0.log" Jul 15 19:39:00.082281 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.082254 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526_60e0d4e3-2398-41f3-a06e-c3c3afaa5022/util/0.log" Jul 15 19:39:00.107225 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.107197 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l5526_60e0d4e3-2398-41f3-a06e-c3c3afaa5022/pull/0.log" Jul 15 19:39:00.137934 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.137904 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n_9fc5d28f-47ad-4c15-a291-210c00fc1cd7/extract/0.log" Jul 15 19:39:00.161764 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.161728 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n_9fc5d28f-47ad-4c15-a291-210c00fc1cd7/util/0.log" Jul 15 19:39:00.187637 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.187609 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734qn2n_9fc5d28f-47ad-4c15-a291-210c00fc1cd7/pull/0.log" Jul 15 19:39:00.239256 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.239173 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s_77f8dc6a-3ad1-4dad-a529-a2d9616e8274/extract/0.log" Jul 15 19:39:00.264552 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.264525 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s_77f8dc6a-3ad1-4dad-a529-a2d9616e8274/util/0.log" Jul 15 19:39:00.290593 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.290554 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c6m5s_77f8dc6a-3ad1-4dad-a529-a2d9616e8274/pull/0.log" Jul 15 19:39:00.729807 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.729043 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7964cb565-h9ffr_c5c79352-4fa4-48e2-86f3-a754c08e393d/authorino/0.log" Jul 15 19:39:00.844034 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:00.844003 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rmv8k_adbc93c7-471f-4406-beeb-e748d9cee3e8/kuadrant-console-plugin/0.log" Jul 15 19:39:03.341049 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:03.341012 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5twct_33a58e62-34c0-41e9-afb1-84b6615367d0/node-exporter/0.log" Jul 15 19:39:03.372689 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:03.372660 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5twct_33a58e62-34c0-41e9-afb1-84b6615367d0/kube-rbac-proxy/0.log" Jul 15 19:39:03.398253 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:03.398222 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5twct_33a58e62-34c0-41e9-afb1-84b6615367d0/init-textfile/0.log" Jul 15 19:39:03.534126 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:03.534093 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-brk9m_4f43eb57-13dd-48fa-b59c-e85ce1a54f5c/kube-rbac-proxy-main/0.log" Jul 15 19:39:03.558311 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:03.558246 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-brk9m_4f43eb57-13dd-48fa-b59c-e85ce1a54f5c/kube-rbac-proxy-self/0.log" Jul 15 19:39:03.581157 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:03.581075 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-brk9m_4f43eb57-13dd-48fa-b59c-e85ce1a54f5c/openshift-state-metrics/0.log" Jul 15 19:39:03.821891 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:03.821848 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-5jkgt_af09c313-fdac-4419-b281-c4e673280f09/prometheus-operator/0.log" Jul 15 19:39:03.844583 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:03.844448 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-5jkgt_af09c313-fdac-4419-b281-c4e673280f09/kube-rbac-proxy/0.log" Jul 15 19:39:03.919624 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:03.919575 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-597fd459d4-9n88f_a068110a-1711-4e92-8a2c-948a08a088d8/telemeter-client/0.log" Jul 15 19:39:03.946666 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:03.946624 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-597fd459d4-9n88f_a068110a-1711-4e92-8a2c-948a08a088d8/reload/0.log" Jul 15 19:39:03.973094 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:03.973068 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-597fd459d4-9n88f_a068110a-1711-4e92-8a2c-948a08a088d8/kube-rbac-proxy/0.log" Jul 15 19:39:05.811731 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:05.811685 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n"] Jul 15 19:39:05.817449 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:05.817413 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:05.836523 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:05.836489 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n"] Jul 15 19:39:05.998001 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:05.997960 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-podres\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:05.998197 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:05.998047 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-sys\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:05.998197 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:05.998088 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-lib-modules\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:05.998197 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:05.998117 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxd4t\" (UniqueName: \"kubernetes.io/projected/1fa4df16-74da-4c5f-b071-5bbe52b38b70-kube-api-access-xxd4t\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:05.998197 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:05.998163 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-proc\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.099723 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.099628 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-proc\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.100046 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.100017 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-podres\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.100918 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.100157 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-podres\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.100918 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.100266 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-sys\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.100918 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.100315 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-lib-modules\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.100918 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.100353 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxd4t\" (UniqueName: \"kubernetes.io/projected/1fa4df16-74da-4c5f-b071-5bbe52b38b70-kube-api-access-xxd4t\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.100918 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.100772 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-sys\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.100918 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.099817 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-proc\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.100918 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.100922 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fa4df16-74da-4c5f-b071-5bbe52b38b70-lib-modules\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.109387 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.109356 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxd4t\" (UniqueName: \"kubernetes.io/projected/1fa4df16-74da-4c5f-b071-5bbe52b38b70-kube-api-access-xxd4t\") pod \"perf-node-gather-daemonset-t254n\" (UID: \"1fa4df16-74da-4c5f-b071-5bbe52b38b70\") " pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.133013 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.132984 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:06.321523 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.321482 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n"] Jul 15 19:39:06.563858 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.563826 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5db54c446c-jl5kh_8cb769cf-e4ed-4f02-8c31-49e1b636f5e0/console/0.log" Jul 15 19:39:06.602552 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.602469 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-869wl_65c16fea-c4fb-4fe2-8199-96ad95a08e38/download-server/0.log" Jul 15 19:39:06.687196 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.687154 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" event={"ID":"1fa4df16-74da-4c5f-b071-5bbe52b38b70","Type":"ContainerStarted","Data":"c80cf229db7271892024994c407082ff7b1b87aea1433bd394b9070d01eb4b6c"} Jul 15 19:39:06.687386 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.687230 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" event={"ID":"1fa4df16-74da-4c5f-b071-5bbe52b38b70","Type":"ContainerStarted","Data":"a97f3ea3c06ae13afa638df7a64f4ab93e719522012b7e5b025af7144d4d3e17"} Jul 15 19:39:06.687386 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:06.687269 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:08.376740 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:08.376713 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qtljf_788b008d-e5d4-46bb-b45d-ae0063bdd5a5/dns/0.log" Jul 15 19:39:08.401266 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:08.401237 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qtljf_788b008d-e5d4-46bb-b45d-ae0063bdd5a5/kube-rbac-proxy/0.log" Jul 15 19:39:08.454160 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:08.454132 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nsvqx_72394437-d6bc-4577-8e92-ce60aec0f34a/dns-node-resolver/0.log" Jul 15 19:39:09.090736 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:09.090700 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-68965cfc66-dz4zt_48b8c77a-430f-4480-940e-0c6d33f5426f/registry/0.log" Jul 15 19:39:09.114886 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:09.114852 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2mlqx_0673f240-f3b2-45cc-bfe2-fe7d49b5cd04/node-ca/0.log" Jul 15 19:39:10.273563 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:10.273528 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbd849f4d-98bjb_596dd6fd-f306-4cdf-ba48-3f73028221c0/kube-auth-proxy/0.log" Jul 15 19:39:10.428683 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:10.428642 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-22jvw_0827f76e-ef0a-44c1-87b9-8ee6eef709be/payload-processing/0.log" Jul 15 19:39:11.295723 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:11.295695 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mdpvq_dd598c73-3e74-4fdb-8367-5360f9ef28fd/serve-healthcheck-canary/0.log" Jul 15 19:39:11.967056 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:11.967027 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hltxx_b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c/kube-rbac-proxy/0.log" Jul 15 19:39:11.996374 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:11.996345 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hltxx_b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c/exporter/0.log" Jul 15 19:39:12.020231 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:12.020206 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hltxx_b07ce01b-9fbe-41c9-bb7b-e8eb58fde37c/extractor/0.log" Jul 15 19:39:12.707254 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:12.707221 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" Jul 15 19:39:12.726479 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:12.726417 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6vp87/perf-node-gather-daemonset-t254n" podStartSLOduration=7.726397513 podStartE2EDuration="7.726397513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:39:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:39:06.704730079 +0000 UTC m=+2897.052442378" watchObservedRunningTime="2026-07-15 19:39:12.726397513 +0000 UTC m=+2903.074109815" Jul 15 19:39:14.802246 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:14.802212 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7dcfcc7558-xnh2k_180331a3-7b84-4974-8c02-e91cc5148a9b/manager/0.log" Jul 15 19:39:14.827195 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:14.827158 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cfwq4_587d6702-7ab8-425d-afc1-3e60ab776316/server/0.log" Jul 15 19:39:14.917530 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:14.917491 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-d7995_d5cf65fc-bc88-4162-8f42-19760b3b9be3/manager/0.log" Jul 15 19:39:15.152511 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:15.152416 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dbd7b8848-wd9vw_12951936-1742-485e-a2a2-7d3b57a86f88/manager/0.log" Jul 15 19:39:15.172242 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:15.172205 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dbd7b8848-wd9vw_12951936-1742-485e-a2a2-7d3b57a86f88/copy-manifests/0.log" Jul 15 19:39:16.233481 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:16.233431 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:39:16.451291 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:16.451205 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7db65df97c-mj9pz_881ce289-489e-40d4-bfe1-4d9720c4601f/manager/0.log" Jul 15 19:39:16.885951 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:16.885923 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbd849f4d-98bjb_596dd6fd-f306-4cdf-ba48-3f73028221c0/kube-auth-proxy/0.log" Jul 15 19:39:16.995281 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:16.995245 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-22jvw_0827f76e-ef0a-44c1-87b9-8ee6eef709be/payload-processing/0.log" Jul 15 19:39:18.331123 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:18.331075 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbd849f4d-98bjb_596dd6fd-f306-4cdf-ba48-3f73028221c0/kube-auth-proxy/0.log" Jul 15 19:39:18.460261 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:18.460221 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-22jvw_0827f76e-ef0a-44c1-87b9-8ee6eef709be/payload-processing/0.log" Jul 15 19:39:18.509183 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:18.509148 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024 Jul 15 19:39:18.609414 ip-10-0-137-11 kubenswrapper[2574]: I0715 19:39:18.609289 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5k4q_7207eef2-b107-462b-bcfb-6302820c4835/konnectivity-agent/0.log" limit=1024