Jul 23 17:59:37.343102 ip-10-0-129-187 systemd[1]: Starting Kubernetes Kubelet... Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.735044 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743943 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743957 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743961 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743964 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743967 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743970 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743973 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:59:37.762072 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743976 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743979 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743981 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743984 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743987 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743990 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743993 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743996 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.743999 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744001 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744004 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744006 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744009 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744011 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744014 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744018 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744020 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744023 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744025 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744028 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:59:37.764362 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744031 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744034 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744036 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744038 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744041 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744043 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744046 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744048 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744051 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744053 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744056 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744059 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744063 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744067 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744071 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744074 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744076 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744079 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744082 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:59:37.766743 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744084 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744087 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744090 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744092 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744095 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744098 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744101 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744103 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744107 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744109 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744112 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744114 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744117 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744119 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744122 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744124 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744127 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744129 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744131 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:59:37.767592 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744134 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744137 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744139 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744142 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744145 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744147 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744149 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744152 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744155 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744157 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744159 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744162 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744166 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744169 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744172 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744174 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744177 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744179 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744182 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744184 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:59:37.769798 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744187 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744189 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744193 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744197 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744200 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744203 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744206 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744209 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744212 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744215 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744217 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744219 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744222 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744224 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744948 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744956 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744958 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744961 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744964 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:59:37.771519 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744967 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744970 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744973 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744976 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744978 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744982 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744984 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744987 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744990 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744993 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744995 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.744998 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745000 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745003 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745006 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745009 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745012 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745015 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745018 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745020 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:59:37.772462 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745022 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745025 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745027 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745030 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745034 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745038 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745040 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745044 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745046 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745049 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745051 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745053 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745056 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745060 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745063 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745066 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745068 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745071 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745073 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:59:37.773321 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745075 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745078 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745080 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745083 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745085 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745088 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745090 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745093 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745095 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745098 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745101 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745103 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745106 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745108 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745111 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745113 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745116 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745118 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745122 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745126 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:59:37.773979 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745129 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745132 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745134 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745136 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745140 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745142 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745145 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745148 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745151 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745154 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745156 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745159 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745161 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745164 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745166 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745169 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745172 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745174 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745177 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:59:37.774616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745182 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745184 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745187 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745190 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745192 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745195 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745197 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745200 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745202 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745204 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745209 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745212 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745214 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745217 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745219 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745221 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745592 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745597 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745602 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:59:37.775231 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745606 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745609 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745612 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745615 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745618 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745620 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745623 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745625 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745627 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745630 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745632 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745634 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745637 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745641 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745643 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745646 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745648 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745651 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745653 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745655 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:59:37.776048 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745658 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745661 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745663 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745667 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745669 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745671 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745675 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745678 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745681 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745684 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745687 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745690 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745692 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745695 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745697 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745700 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745703 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745705 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745708 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:59:37.776668 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745710 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745712 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745715 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745717 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745720 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745722 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745726 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745728 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745731 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745733 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745736 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745738 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745741 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745743 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745746 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745748 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745752 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745754 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745757 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745760 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:59:37.777285 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745762 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745765 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745767 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745770 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745773 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745775 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745777 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745780 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745783 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745785 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745788 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745790 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745792 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745795 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745797 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745816 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745820 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745823 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745826 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:59:37.777891 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745829 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745831 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745835 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745837 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745840 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745843 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745846 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745849 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745851 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745855 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745857 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745860 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745863 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745866 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745868 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745871 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745873 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.745876 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745943 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745950 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 17:59:37.778414 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745956 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745960 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745965 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745968 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745973 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745977 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745980 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745983 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745986 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745990 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745993 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.745998 2562 flags.go:64] FLAG: --cgroup-root="" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746001 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746004 2562 flags.go:64] FLAG: --client-ca-file="" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746006 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746009 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746014 2562 flags.go:64] FLAG: --cluster-domain="" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746017 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746020 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746023 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746026 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746032 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746035 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 17:59:37.779286 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746038 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746041 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746044 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746047 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746049 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746052 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746055 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746059 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746062 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746065 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746067 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746070 2562 flags.go:64] FLAG: --enable-server="true" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746073 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746078 2562 flags.go:64] FLAG: --event-burst="100" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746081 2562 flags.go:64] FLAG: --event-qps="50" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746084 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746087 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746089 2562 flags.go:64] FLAG: --eviction-hard="" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746093 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746096 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746100 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746103 2562 flags.go:64] FLAG: --eviction-soft="" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746106 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746109 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746112 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 17:59:37.780049 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746114 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746117 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746120 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746123 2562 flags.go:64] FLAG: --feature-gates="" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746126 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746133 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746136 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746139 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746142 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746145 2562 flags.go:64] FLAG: --help="false" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746147 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-129-187.ec2.internal" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746150 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746153 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746156 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746159 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746163 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746166 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746168 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746171 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746174 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746177 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746180 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746183 2562 flags.go:64] FLAG: --kube-reserved="" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746186 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 17:59:37.781122 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746189 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746192 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746194 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746198 2562 flags.go:64] FLAG: --lock-file="" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746201 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746204 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746207 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746212 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746215 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746217 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746220 2562 flags.go:64] FLAG: --logging-format="text" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746223 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746226 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746229 2562 flags.go:64] FLAG: --manifest-url="" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746232 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746236 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746239 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746243 2562 flags.go:64] FLAG: --max-pods="110" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746246 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746249 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746252 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746254 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746257 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746260 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746263 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 17:59:37.782303 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746270 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746273 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746276 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746279 2562 flags.go:64] FLAG: --pod-cidr="" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746283 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746288 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746291 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746294 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746296 2562 flags.go:64] FLAG: --port="10250" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746300 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746302 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07fcc930320ac6d4c" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746306 2562 flags.go:64] FLAG: --qos-reserved="" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746309 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746312 2562 flags.go:64] FLAG: --register-node="true" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746315 2562 flags.go:64] FLAG: --register-with-taints="" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746318 2562 flags.go:64] FLAG: --registry-burst="10" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746321 2562 flags.go:64] FLAG: --registry-qps="5" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746323 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746326 2562 flags.go:64] FLAG: --reserved-memory="" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746330 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746334 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746337 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746340 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746343 2562 flags.go:64] FLAG: --runonce="false" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746345 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 17:59:37.783053 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746348 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746351 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746353 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746356 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746359 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746362 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746365 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746368 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746371 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746373 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746376 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746379 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746382 2562 flags.go:64] FLAG: --system-cgroups="" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746385 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746390 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746392 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746395 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746399 2562 flags.go:64] FLAG: --tls-min-version="" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746403 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746406 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746409 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746412 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746414 2562 flags.go:64] FLAG: --v="2" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746423 2562 flags.go:64] FLAG: --version="false" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746427 2562 flags.go:64] FLAG: --vmodule="" Jul 23 17:59:37.866646 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746430 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 17:59:37.831994 ip-10-0-129-187 systemd[1]: Started Kubernetes Kubelet. Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.746433 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746547 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746551 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746555 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746558 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746560 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746562 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746565 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746568 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746570 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746573 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746575 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746578 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746580 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746583 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746585 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746587 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746590 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746592 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:59:37.867593 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746595 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746598 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746600 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746603 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746605 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746610 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746614 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746617 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746620 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746623 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746625 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746628 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746630 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746633 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746636 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746639 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746643 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746646 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746649 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:59:37.868616 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746652 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746655 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746657 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746660 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746663 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746666 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746668 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746671 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746673 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746676 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746678 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746680 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746683 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746685 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746688 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746690 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746693 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746696 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746699 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746736 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:59:37.869419 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746741 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746745 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746748 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746752 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746756 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746759 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746761 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746764 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746767 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746769 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746772 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746775 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746777 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746780 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746783 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746785 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746788 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746792 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746795 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:59:37.870322 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746798 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746814 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746818 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746820 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746823 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746826 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746828 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746832 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746834 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746837 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746843 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746847 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746850 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746857 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746860 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746863 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746865 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746868 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746870 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:59:37.871137 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746873 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746875 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746878 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.746881 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.747439 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.754421 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.754437 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.754455 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.754461 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754529 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754533 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754537 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754539 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754542 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754544 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:59:37.871745 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754547 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754550 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754553 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754556 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754559 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754561 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754564 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754566 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754569 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754572 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754575 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754583 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754586 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754589 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754592 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754594 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754596 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754599 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754602 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754604 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:59:37.872408 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754607 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754609 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754612 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754615 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754617 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754619 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754622 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754624 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754627 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754629 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754632 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754634 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754636 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754639 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754643 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754646 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754649 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754652 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754655 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754658 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:59:37.873152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754660 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754667 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754670 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754672 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754681 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754685 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754688 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754691 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754694 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754696 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754699 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754702 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754705 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754708 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754710 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754713 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754715 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754718 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754720 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:59:37.873788 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754723 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754725 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754728 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754731 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754733 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754736 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754738 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754740 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754743 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754745 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754748 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754750 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754753 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754755 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754760 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754762 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754765 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754767 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754777 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754779 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:59:37.874350 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754782 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754786 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754789 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754791 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754794 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754797 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754814 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754819 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754823 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754825 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754828 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754831 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754833 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754836 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:59:37.874927 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.754841 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754981 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754985 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754988 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754991 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754994 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754997 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.754999 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755002 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755004 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755006 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755009 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755013 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755016 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755019 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755022 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755024 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755034 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755037 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755039 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:59:37.875302 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755041 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755044 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755046 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755049 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755051 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755054 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755056 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755058 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755061 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755064 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755066 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755068 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755071 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755073 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755075 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755078 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755080 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755083 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755085 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755088 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:59:37.875880 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755090 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755092 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755095 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755098 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755101 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755105 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755107 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755110 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755112 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755120 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755128 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755131 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755133 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755136 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755140 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755143 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755146 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755149 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755151 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:59:37.876609 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755154 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755157 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755159 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755162 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755164 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755166 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755169 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755171 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755174 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755176 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755179 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755181 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755184 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755186 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755189 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755191 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755193 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755197 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755202 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755204 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:59:37.877467 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755206 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755209 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755211 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755214 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755221 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755224 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755226 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755229 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755231 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755234 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755236 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755238 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755241 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755243 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755245 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755248 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755250 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755253 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755257 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:59:37.878152 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755261 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:37.755264 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.755269 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.755394 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.757373 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.759288 2562 server.go:1019] "Starting client certificate rotation" Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.761532 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.761568 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.761603 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.781655 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.791417 2562 log.go:25] "Validated CRI v1 runtime API" Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.796615 2562 log.go:25] "Validated CRI v1 image API" Jul 23 17:59:37.878780 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.797409 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 17:59:37.879398 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.799647 2562 fs.go:135] Filesystem UUIDs: map[2a22b8fe-f332-4f9e-a47e-f6f03d97456e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f25ca354-128e-4980-80e9-5d19964fb25d:/dev/nvme0n1p4] Jul 23 17:59:37.879398 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.799665 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.805343 2562 manager.go:216] Machine: {Timestamp:2026-07-23 17:59:37.804094695 +0000 UTC m=+0.349994991 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3198080 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2714b2c1811df3f17ba9149193a15f SystemUUID:ec2714b2-c181-1df3-f17b-a9149193a15f BootID:d20c3a8e-7c5b-41e1-b0ae-717f54d828fa Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3e:89:54:0e:bb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3e:89:54:0e:bb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:78:76:c2:d6:41 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.805444 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.805525 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.808282 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.808319 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-187.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.809675 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.809689 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.809699 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.810185 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.811379 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.814635 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.814776 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.815485 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.815499 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.815524 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.815537 2562 kubelet.go:398] "Adding apiserver pod source" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.815550 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 17:59:37.884651 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.816021 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.816041 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.818086 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.822492 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.822879 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.823332 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824694 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824711 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824717 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824725 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824736 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824743 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824751 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824757 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824764 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824770 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824779 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.824784 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.825793 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.825822 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.827743 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-187.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.827749 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.831386 2562 server.go:1268] "Started kubelet" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.831715 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.832614 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.833362 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.833395 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.834531 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.839037 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.840501 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-187.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.840724 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.840729 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.840781 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-187.ec2.internal\" not found" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841382 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841453 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841465 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.840513 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-187.ec2.internal.18c4fd0b6b3cb4c4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-187.ec2.internal,UID:ip-10-0-129-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-187.ec2.internal,},FirstTimestamp:2026-07-23 17:59:37.831167172 +0000 UTC m=+0.377067468,LastTimestamp:2026-07-23 17:59:37.831167172 +0000 UTC m=+0.377067468,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-187.ec2.internal,}" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841494 2562 factory.go:55] Registering systemd factory Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841538 2562 factory.go:223] Registration of the systemd container factory successfully Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841566 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 23 17:59:37.885473 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841576 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841833 2562 factory.go:153] Registering CRI-O factory Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841848 2562 factory.go:223] Registration of the crio container factory successfully Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841902 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841925 2562 factory.go:103] Registering Raw factory Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.841941 2562 manager.go:1196] Started watching for new ooms in manager Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.842323 2562 manager.go:349] Starting recovery of all containers Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.843415 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-njzng" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.843898 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-187.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.844108 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.851730 2562 manager.go:354] Recovery completed Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.856527 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.858702 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.858727 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.858736 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.859250 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.859261 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.859266 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.859275 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.860035 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.860063 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.860074 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.860631 2562 policy_none.go:49] "None policy: Start" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.860648 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.860662 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.861347 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-187.ec2.internal.18c4fd0b6ce10a01 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-187.ec2.internal,UID:ip-10-0-129-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-187.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-187.ec2.internal,},FirstTimestamp:2026-07-23 17:59:37.858714113 +0000 UTC m=+0.404614413,LastTimestamp:2026-07-23 17:59:37.858714113 +0000 UTC m=+0.404614413,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-187.ec2.internal,}" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.862075 2562 policy_none.go:47] "Start" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.872462 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-187.ec2.internal.18c4fd0b6ce14ccd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-187.ec2.internal,UID:ip-10-0-129-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-187.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-187.ec2.internal,},FirstTimestamp:2026-07-23 17:59:37.858731213 +0000 UTC m=+0.404631510,LastTimestamp:2026-07-23 17:59:37.858731213 +0000 UTC m=+0.404631510,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-187.ec2.internal,}" Jul 23 17:59:37.999969 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.872907 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-njzng" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.881643 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-187.ec2.internal.18c4fd0b6ce16e00 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-187.ec2.internal,UID:ip-10-0-129-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-187.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-187.ec2.internal,},FirstTimestamp:2026-07-23 17:59:37.858739712 +0000 UTC m=+0.404640009,LastTimestamp:2026-07-23 17:59:37.858739712 +0000 UTC m=+0.404640009,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-187.ec2.internal,}" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.897394 2562 manager.go:337] "Starting Device Plugin manager" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.897428 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.897436 2562 server.go:87] "Starting device plugin registration server" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.897671 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.897685 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.897773 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.897910 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.897918 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.898340 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:37.898389 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-187.ec2.internal\" not found" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.998058 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.999023 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.999047 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.999058 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:37.999083 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-187.ec2.internal" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:38.011044 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-187.ec2.internal" Jul 23 17:59:38.017596 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:38.011068 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-187.ec2.internal\": node \"ip-10-0-129-187.ec2.internal\" not found" Jul 23 17:59:39.214801 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:38.064581 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 17:59:39.214801 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:38.065743 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 17:59:39.214801 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:38.065771 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 17:59:39.214801 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:38.065792 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 17:59:39.214801 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:38.065837 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 17:59:39.214801 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:38.068686 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:59:39.214801 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:38.087750 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-187.ec2.internal\" not found" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.224866 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-187.ec2.internal"] Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.225104 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.226039 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.226060 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.226071 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.227486 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.227635 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.227664 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.228204 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.228235 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.228253 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.228253 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.228268 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.228278 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.229593 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.229615 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.230361 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.230387 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.230402 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.246863 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-187.ec2.internal\" not found" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.266906 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-187.ec2.internal\" not found" node="ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.271178 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-187.ec2.internal\" not found" node="ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.302256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/116152a5548a914d4afdfde51aad9a78-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal\" (UID: \"116152a5548a914d4afdfde51aad9a78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.302282 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/116152a5548a914d4afdfde51aad9a78-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal\" (UID: \"116152a5548a914d4afdfde51aad9a78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.342300 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.302307 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/12a6046fa193489a163cd70c7bb12503-config\") pod \"kube-apiserver-proxy-ip-10-0-129-187.ec2.internal\" (UID: \"12a6046fa193489a163cd70c7bb12503\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.467149 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.347767 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-187.ec2.internal\" not found" Jul 23 17:59:39.467149 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.402774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/116152a5548a914d4afdfde51aad9a78-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal\" (UID: \"116152a5548a914d4afdfde51aad9a78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.467149 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.402798 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/12a6046fa193489a163cd70c7bb12503-config\") pod \"kube-apiserver-proxy-ip-10-0-129-187.ec2.internal\" (UID: \"12a6046fa193489a163cd70c7bb12503\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.467149 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.402813 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/116152a5548a914d4afdfde51aad9a78-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal\" (UID: \"116152a5548a914d4afdfde51aad9a78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.467149 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.402840 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/116152a5548a914d4afdfde51aad9a78-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal\" (UID: \"116152a5548a914d4afdfde51aad9a78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.467149 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.402881 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/116152a5548a914d4afdfde51aad9a78-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal\" (UID: \"116152a5548a914d4afdfde51aad9a78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.467149 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.402902 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/12a6046fa193489a163cd70c7bb12503-config\") pod \"kube-apiserver-proxy-ip-10-0-129-187.ec2.internal\" (UID: \"12a6046fa193489a163cd70c7bb12503\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.467149 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.448677 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-187.ec2.internal\" not found" Jul 23 17:59:39.572897 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.549163 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-187.ec2.internal\" not found" Jul 23 17:59:39.572897 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.570258 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.576017 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.574381 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.694765 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.649919 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-187.ec2.internal\" not found" Jul 23 17:59:39.841782 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.750482 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-187.ec2.internal\" not found" Jul 23 17:59:39.841782 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.757461 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:59:39.841782 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.792724 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:59:39.841782 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.801282 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.841782 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.816665 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 17:59:39.841782 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.816777 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 17:59:39.841782 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.816819 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 17:59:39.841782 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.816825 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 17:59:39.841782 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.816857 2562 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a11f9e5ab929f4d7283be086db220920-4009fd6c4839e6b0.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.129.187:35994->44.196.0.178:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.841782 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.816886 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-187.ec2.internal" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.868405 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.874583 2562 apiserver.go:52] "Watching apiserver" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.898401 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.899222 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.901631 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-45szx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal","openshift-multus/multus-additional-cni-plugins-5m2x2","openshift-multus/network-metrics-daemon-88l5f","openshift-network-diagnostics/network-check-target-l8nhz","openshift-ovn-kubernetes/ovnkube-node-s7stv","kube-system/kube-apiserver-proxy-ip-10-0-129-187.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq","openshift-cluster-node-tuning-operator/tuned-654sp","openshift-dns/node-resolver-ngf5j","openshift-image-registry/node-ca-ljb4k","openshift-multus/multus-mjcqv","openshift-network-operator/iptables-alerter-b8c5k","kube-system/global-pull-secret-syncer-h62kj"] Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.903487 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-45szx" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.906823 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.907405 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.907455 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.907750 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.907864 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.907896 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.908787 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2cv5h\"" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.908825 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.909292 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.910828 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.911977 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.912548 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.913183 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ngf5j" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.914690 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ljb4k" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.916134 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mjcqv" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.917882 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b8c5k" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.918143 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.918933 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.919045 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.919148 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.919366 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.919375 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:39.919417 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 17:59:39.972958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.928557 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.931325 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.931582 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.931672 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.931931 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.931935 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932095 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kb9x4\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932104 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kddb5\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932128 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932177 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932186 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7dxcl\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932183 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932246 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932348 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gh5p5\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932373 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6l4xb\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932449 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932859 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932871 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jcpl2\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932884 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rw676\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932936 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x2d9n\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932886 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.932873 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.935013 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.935051 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.935070 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.935283 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.935824 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.939994 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 17:54:38 +0000 UTC" deadline="2028-03-26 05:10:55.807851643 +0000 UTC" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.940025 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14675h11m15.86782854s" Jul 23 17:59:39.974470 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:39.961928 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.001999 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005053 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-run\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005082 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14934d93-f8ca-478d-ba2d-24c70d9ab9ec-konnectivity-ca\") pod \"konnectivity-agent-45szx\" (UID: \"14934d93-f8ca-478d-ba2d-24c70d9ab9ec\") " pod="kube-system/konnectivity-agent-45szx" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-cni-netd\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b673b420-6fcb-4021-8ea7-1dce6101f1ad-tmp-dir\") pod \"node-resolver-ngf5j\" (UID: \"b673b420-6fcb-4021-8ea7-1dce6101f1ad\") " pod="openshift-dns/node-resolver-ngf5j" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005161 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-run-k8s-cni-cncf-io\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lclp9\" (UniqueName: \"kubernetes.io/projected/106e79cf-58a7-4cfd-9738-01fa20d637f1-kube-api-access-lclp9\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005205 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b8945dd9-218d-4d58-b42e-6c2ca865e382-host-slash\") pod \"iptables-alerter-b8c5k\" (UID: \"b8945dd9-218d-4d58-b42e-6c2ca865e382\") " pod="openshift-network-operator/iptables-alerter-b8c5k" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14934d93-f8ca-478d-ba2d-24c70d9ab9ec-agent-certs\") pod \"konnectivity-agent-45szx\" (UID: \"14934d93-f8ca-478d-ba2d-24c70d9ab9ec\") " pod="kube-system/konnectivity-agent-45szx" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005268 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5350fb76-01c2-43f7-bd40-29c9b0622283-tmp\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005301 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005323 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b816e2d-d250-4c9f-abb2-ff37218ff860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005342 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f103464-e957-4cb4-bd09-c0361841a2b5-ovnkube-script-lib\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-kubernetes\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005371 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-sysctl-d\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005385 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-var-lib-kubelet\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.089012 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005400 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005423 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh2hb\" (UniqueName: \"kubernetes.io/projected/daeb5059-d59d-457e-a5c5-d8545f19b19e-kube-api-access-wh2hb\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005440 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005453 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-cnibin\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005467 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-os-release\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005480 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/106e79cf-58a7-4cfd-9738-01fa20d637f1-cni-binary-copy\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005516 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005541 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-run-netns\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005623 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-socket-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005648 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-socket-dir-parent\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005673 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8x4r\" (UniqueName: \"kubernetes.io/projected/b8945dd9-218d-4d58-b42e-6c2ca865e382-kube-api-access-d8x4r\") pod \"iptables-alerter-b8c5k\" (UID: \"b8945dd9-218d-4d58-b42e-6c2ca865e382\") " pod="openshift-network-operator/iptables-alerter-b8c5k" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005689 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-system-cni-dir\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005704 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b816e2d-d250-4c9f-abb2-ff37218ff860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005719 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f103464-e957-4cb4-bd09-c0361841a2b5-ovn-node-metrics-cert\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005734 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-etc-selinux\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.090181 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005749 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b673b420-6fcb-4021-8ea7-1dce6101f1ad-hosts-file\") pod \"node-resolver-ngf5j\" (UID: \"b673b420-6fcb-4021-8ea7-1dce6101f1ad\") " pod="openshift-dns/node-resolver-ngf5j" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-hostroot\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005810 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-run-multus-certs\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005833 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qbw8x" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005838 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/822c461b-5e7b-4abb-a090-edacc6b4392c-kubelet-config\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005869 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-var-lib-cni-bin\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005893 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-daemon-config\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-node-log\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005945 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-cni-bin\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.005972 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b939e580-7ce8-4e0b-a2c6-6e55f65e25d8-host\") pod \"node-ca-ljb4k\" (UID: \"b939e580-7ce8-4e0b-a2c6-6e55f65e25d8\") " pod="openshift-image-registry/node-ca-ljb4k" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006022 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtt5w\" (UniqueName: \"kubernetes.io/projected/b939e580-7ce8-4e0b-a2c6-6e55f65e25d8-kube-api-access-mtt5w\") pod \"node-ca-ljb4k\" (UID: \"b939e580-7ce8-4e0b-a2c6-6e55f65e25d8\") " pod="openshift-image-registry/node-ca-ljb4k" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-run-netns\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006077 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-var-lib-kubelet\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006103 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f103464-e957-4cb4-bd09-c0361841a2b5-ovnkube-config\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-run-openvswitch\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006132 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-modprobe-d\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006146 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-tuned\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.091206 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006160 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b939e580-7ce8-4e0b-a2c6-6e55f65e25d8-serviceca\") pod \"node-ca-ljb4k\" (UID: \"b939e580-7ce8-4e0b-a2c6-6e55f65e25d8\") " pod="openshift-image-registry/node-ca-ljb4k" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006204 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006251 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-cni-dir\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006271 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-var-lib-cni-multus\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006291 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-kubelet\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006313 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-sysconfig\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-sysctl-conf\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006366 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-systemd\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006383 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-host\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006400 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-device-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006420 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-sys-fs\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006441 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlktv\" (UniqueName: \"kubernetes.io/projected/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-kube-api-access-vlktv\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006459 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-etc-kubernetes\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006480 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-etc-openvswitch\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006500 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-run-ovn\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-registration-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.092493 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006530 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfwnm\" (UniqueName: \"kubernetes.io/projected/b673b420-6fcb-4021-8ea7-1dce6101f1ad-kube-api-access-pfwnm\") pod \"node-resolver-ngf5j\" (UID: \"b673b420-6fcb-4021-8ea7-1dce6101f1ad\") " pod="openshift-dns/node-resolver-ngf5j" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006549 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b8945dd9-218d-4d58-b42e-6c2ca865e382-iptables-alerter-script\") pod \"iptables-alerter-b8c5k\" (UID: \"b8945dd9-218d-4d58-b42e-6c2ca865e382\") " pod="openshift-network-operator/iptables-alerter-b8c5k" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-os-release\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006583 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b816e2d-d250-4c9f-abb2-ff37218ff860-cni-binary-copy\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006596 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-systemd-units\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-lib-modules\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006622 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8697\" (UniqueName: \"kubernetes.io/projected/5350fb76-01c2-43f7-bd40-29c9b0622283-kube-api-access-v8697\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-cnibin\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006662 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np8b7\" (UniqueName: \"kubernetes.io/projected/1b816e2d-d250-4c9f-abb2-ff37218ff860-kube-api-access-np8b7\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006676 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-run-systemd\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006715 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-var-lib-openvswitch\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006731 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/822c461b-5e7b-4abb-a090-edacc6b4392c-dbus\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-system-cni-dir\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006759 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f103464-e957-4cb4-bd09-c0361841a2b5-env-overrides\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006777 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-conf-dir\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006802 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fhbm\" (UniqueName: \"kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm\") pod \"network-check-target-l8nhz\" (UID: \"4178e19f-ad0f-4366-8f06-471cb052caf5\") " pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:40.093485 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006815 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-log-socket\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.095019 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006830 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-sys\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.095019 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-slash\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.095019 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.006861 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjxsv\" (UniqueName: \"kubernetes.io/projected/9f103464-e957-4cb4-bd09-c0361841a2b5-kube-api-access-gjxsv\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.095019 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.024534 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qbw8x" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-os-release\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107702 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b816e2d-d250-4c9f-abb2-ff37218ff860-cni-binary-copy\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107716 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-systemd-units\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-lib-modules\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107804 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-systemd-units\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107824 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-lib-modules\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107823 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-os-release\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107863 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8697\" (UniqueName: \"kubernetes.io/projected/5350fb76-01c2-43f7-bd40-29c9b0622283-kube-api-access-v8697\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107882 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-cnibin\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107899 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np8b7\" (UniqueName: \"kubernetes.io/projected/1b816e2d-d250-4c9f-abb2-ff37218ff860-kube-api-access-np8b7\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107918 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-run-systemd\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-var-lib-openvswitch\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107950 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/822c461b-5e7b-4abb-a090-edacc6b4392c-dbus\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107952 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-cnibin\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107965 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-system-cni-dir\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.107982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f103464-e957-4cb4-bd09-c0361841a2b5-env-overrides\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.174310 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108002 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-conf-dir\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fhbm\" (UniqueName: \"kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm\") pod \"network-check-target-l8nhz\" (UID: \"4178e19f-ad0f-4366-8f06-471cb052caf5\") " pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-log-socket\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-sys\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108080 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-slash\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108099 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjxsv\" (UniqueName: \"kubernetes.io/projected/9f103464-e957-4cb4-bd09-c0361841a2b5-kube-api-access-gjxsv\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108116 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-run\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14934d93-f8ca-478d-ba2d-24c70d9ab9ec-konnectivity-ca\") pod \"konnectivity-agent-45szx\" (UID: \"14934d93-f8ca-478d-ba2d-24c70d9ab9ec\") " pod="kube-system/konnectivity-agent-45szx" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108146 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-cni-netd\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108166 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b673b420-6fcb-4021-8ea7-1dce6101f1ad-tmp-dir\") pod \"node-resolver-ngf5j\" (UID: \"b673b420-6fcb-4021-8ea7-1dce6101f1ad\") " pod="openshift-dns/node-resolver-ngf5j" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108180 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-run-k8s-cni-cncf-io\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108199 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lclp9\" (UniqueName: \"kubernetes.io/projected/106e79cf-58a7-4cfd-9738-01fa20d637f1-kube-api-access-lclp9\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108229 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b8945dd9-218d-4d58-b42e-6c2ca865e382-host-slash\") pod \"iptables-alerter-b8c5k\" (UID: \"b8945dd9-218d-4d58-b42e-6c2ca865e382\") " pod="openshift-network-operator/iptables-alerter-b8c5k" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108246 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14934d93-f8ca-478d-ba2d-24c70d9ab9ec-agent-certs\") pod \"konnectivity-agent-45szx\" (UID: \"14934d93-f8ca-478d-ba2d-24c70d9ab9ec\") " pod="kube-system/konnectivity-agent-45szx" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108261 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5350fb76-01c2-43f7-bd40-29c9b0622283-tmp\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108275 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108273 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-log-socket\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.176099 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b816e2d-d250-4c9f-abb2-ff37218ff860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108312 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f103464-e957-4cb4-bd09-c0361841a2b5-ovnkube-script-lib\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108328 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-kubernetes\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108329 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-run-systemd\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108342 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-sysctl-d\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108357 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-var-lib-kubelet\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108373 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108370 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-var-lib-openvswitch\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108402 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh2hb\" (UniqueName: \"kubernetes.io/projected/daeb5059-d59d-457e-a5c5-d8545f19b19e-kube-api-access-wh2hb\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108413 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b816e2d-d250-4c9f-abb2-ff37218ff860-cni-binary-copy\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108447 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108443 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-cni-netd\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-system-cni-dir\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108458 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b8945dd9-218d-4d58-b42e-6c2ca865e382-host-slash\") pod \"iptables-alerter-b8c5k\" (UID: \"b8945dd9-218d-4d58-b42e-6c2ca865e382\") " pod="openshift-network-operator/iptables-alerter-b8c5k" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108485 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-run-k8s-cni-cncf-io\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108491 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-conf-dir\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108498 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-slash\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108516 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-cnibin\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.176983 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-os-release\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/106e79cf-58a7-4cfd-9738-01fa20d637f1-cni-binary-copy\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-run-netns\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108631 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-run\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108643 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/822c461b-5e7b-4abb-a090-edacc6b4392c-dbus\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108673 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-socket-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108710 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-sysctl-d\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108770 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-socket-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108770 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108815 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-socket-dir-parent\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108831 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-sys\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108847 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8x4r\" (UniqueName: \"kubernetes.io/projected/b8945dd9-218d-4d58-b42e-6c2ca865e382-kube-api-access-d8x4r\") pod \"iptables-alerter-b8c5k\" (UID: \"b8945dd9-218d-4d58-b42e-6c2ca865e382\") " pod="openshift-network-operator/iptables-alerter-b8c5k" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-system-cni-dir\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108895 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b816e2d-d250-4c9f-abb2-ff37218ff860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f103464-e957-4cb4-bd09-c0361841a2b5-ovn-node-metrics-cert\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.178701 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108949 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-etc-selinux\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.108973 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b673b420-6fcb-4021-8ea7-1dce6101f1ad-hosts-file\") pod \"node-resolver-ngf5j\" (UID: \"b673b420-6fcb-4021-8ea7-1dce6101f1ad\") " pod="openshift-dns/node-resolver-ngf5j" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109015 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b673b420-6fcb-4021-8ea7-1dce6101f1ad-hosts-file\") pod \"node-resolver-ngf5j\" (UID: \"b673b420-6fcb-4021-8ea7-1dce6101f1ad\") " pod="openshift-dns/node-resolver-ngf5j" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109049 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-system-cni-dir\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109054 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b816e2d-d250-4c9f-abb2-ff37218ff860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109095 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b673b420-6fcb-4021-8ea7-1dce6101f1ad-tmp-dir\") pod \"node-resolver-ngf5j\" (UID: \"b673b420-6fcb-4021-8ea7-1dce6101f1ad\") " pod="openshift-dns/node-resolver-ngf5j" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109152 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f103464-e957-4cb4-bd09-c0361841a2b5-ovnkube-script-lib\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109157 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-cnibin\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109229 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-os-release\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109271 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-kubernetes\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109511 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f103464-e957-4cb4-bd09-c0361841a2b5-env-overrides\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109526 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b816e2d-d250-4c9f-abb2-ff37218ff860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109559 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b816e2d-d250-4c9f-abb2-ff37218ff860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109597 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-hostroot\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109609 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-run-netns\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109651 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-socket-dir-parent\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109565 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-hostroot\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109669 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-etc-selinux\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.179575 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109682 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-run-multus-certs\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/822c461b-5e7b-4abb-a090-edacc6b4392c-kubelet-config\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109723 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109744 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-var-lib-cni-bin\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109754 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-var-lib-kubelet\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109761 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-run-multus-certs\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-daemon-config\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-node-log\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109802 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/106e79cf-58a7-4cfd-9738-01fa20d637f1-cni-binary-copy\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109810 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/822c461b-5e7b-4abb-a090-edacc6b4392c-kubelet-config\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109820 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-cni-bin\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.109830 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109843 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b939e580-7ce8-4e0b-a2c6-6e55f65e25d8-host\") pod \"node-ca-ljb4k\" (UID: \"b939e580-7ce8-4e0b-a2c6-6e55f65e25d8\") " pod="openshift-image-registry/node-ca-ljb4k" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109862 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-var-lib-cni-bin\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtt5w\" (UniqueName: \"kubernetes.io/projected/b939e580-7ce8-4e0b-a2c6-6e55f65e25d8-kube-api-access-mtt5w\") pod \"node-ca-ljb4k\" (UID: \"b939e580-7ce8-4e0b-a2c6-6e55f65e25d8\") " pod="openshift-image-registry/node-ca-ljb4k" Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.109901 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs podName:daeb5059-d59d-457e-a5c5-d8545f19b19e nodeName:}" failed. No retries permitted until 2026-07-23 17:59:40.609866568 +0000 UTC m=+2.097358538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs") pod "network-metrics-daemon-88l5f" (UID: "daeb5059-d59d-457e-a5c5-d8545f19b19e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:40.181457 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109920 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-cni-bin\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109924 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.109956 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-node-log\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110020 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-run-netns\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110040 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b939e580-7ce8-4e0b-a2c6-6e55f65e25d8-host\") pod \"node-ca-ljb4k\" (UID: \"b939e580-7ce8-4e0b-a2c6-6e55f65e25d8\") " pod="openshift-image-registry/node-ca-ljb4k" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110041 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-var-lib-kubelet\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110064 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-var-lib-kubelet\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110076 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f103464-e957-4cb4-bd09-c0361841a2b5-ovnkube-config\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110091 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-run-netns\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-run-openvswitch\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110116 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-modprobe-d\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-tuned\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110143 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-run-openvswitch\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b939e580-7ce8-4e0b-a2c6-6e55f65e25d8-serviceca\") pod \"node-ca-ljb4k\" (UID: \"b939e580-7ce8-4e0b-a2c6-6e55f65e25d8\") " pod="openshift-image-registry/node-ca-ljb4k" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110184 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110203 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-cni-dir\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110237 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-var-lib-cni-multus\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110257 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-kubelet\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.182663 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110276 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-sysconfig\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110288 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14934d93-f8ca-478d-ba2d-24c70d9ab9ec-konnectivity-ca\") pod \"konnectivity-agent-45szx\" (UID: \"14934d93-f8ca-478d-ba2d-24c70d9ab9ec\") " pod="kube-system/konnectivity-agent-45szx" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-sysctl-conf\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110320 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-systemd\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110341 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-host\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110344 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-host-var-lib-cni-multus\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-device-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-sys-fs\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110413 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-systemd\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlktv\" (UniqueName: \"kubernetes.io/projected/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-kube-api-access-vlktv\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110437 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-device-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110364 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-daemon-config\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110459 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-etc-kubernetes\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110509 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-host\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110556 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-modprobe-d\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110558 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f103464-e957-4cb4-bd09-c0361841a2b5-ovnkube-config\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110574 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-sys-fs\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.110576 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:40.184841 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110597 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-etc-openvswitch\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.110626 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret podName:822c461b-5e7b-4abb-a090-edacc6b4392c nodeName:}" failed. No retries permitted until 2026-07-23 17:59:40.610612269 +0000 UTC m=+2.098104240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret") pod "global-pull-secret-syncer-h62kj" (UID: "822c461b-5e7b-4abb-a090-edacc6b4392c") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110638 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-etc-openvswitch\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110623 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-multus-cni-dir\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-run-ovn\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110661 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-sysconfig\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-registration-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfwnm\" (UniqueName: \"kubernetes.io/projected/b673b420-6fcb-4021-8ea7-1dce6101f1ad-kube-api-access-pfwnm\") pod \"node-resolver-ngf5j\" (UID: \"b673b420-6fcb-4021-8ea7-1dce6101f1ad\") " pod="openshift-dns/node-resolver-ngf5j" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b8945dd9-218d-4d58-b42e-6c2ca865e382-iptables-alerter-script\") pod \"iptables-alerter-b8c5k\" (UID: \"b8945dd9-218d-4d58-b42e-6c2ca865e382\") " pod="openshift-network-operator/iptables-alerter-b8c5k" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110784 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b939e580-7ce8-4e0b-a2c6-6e55f65e25d8-serviceca\") pod \"node-ca-ljb4k\" (UID: \"b939e580-7ce8-4e0b-a2c6-6e55f65e25d8\") " pod="openshift-image-registry/node-ca-ljb4k" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110714 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/106e79cf-58a7-4cfd-9738-01fa20d637f1-etc-kubernetes\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110677 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-run-ovn\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110833 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f103464-e957-4cb4-bd09-c0361841a2b5-host-kubelet\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.110936 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-sysctl-conf\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.111312 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-registration-dir\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.111359 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b8945dd9-218d-4d58-b42e-6c2ca865e382-iptables-alerter-script\") pod \"iptables-alerter-b8c5k\" (UID: \"b8945dd9-218d-4d58-b42e-6c2ca865e382\") " pod="openshift-network-operator/iptables-alerter-b8c5k" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.113161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14934d93-f8ca-478d-ba2d-24c70d9ab9ec-agent-certs\") pod \"konnectivity-agent-45szx\" (UID: \"14934d93-f8ca-478d-ba2d-24c70d9ab9ec\") " pod="kube-system/konnectivity-agent-45szx" Jul 23 17:59:40.185832 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.113325 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5350fb76-01c2-43f7-bd40-29c9b0622283-tmp\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.113428 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f103464-e957-4cb4-bd09-c0361841a2b5-ovn-node-metrics-cert\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.114388 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5350fb76-01c2-43f7-bd40-29c9b0622283-etc-tuned\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.140693 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.140723 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.140755 2562 projected.go:196] Error preparing data for projected volume kube-api-access-4fhbm for pod openshift-network-diagnostics/network-check-target-l8nhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.140953 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm podName:4178e19f-ad0f-4366-8f06-471cb052caf5 nodeName:}" failed. No retries permitted until 2026-07-23 17:59:40.64092135 +0000 UTC m=+2.128413348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4fhbm" (UniqueName: "kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm") pod "network-check-target-l8nhz" (UID: "4178e19f-ad0f-4366-8f06-471cb052caf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.142239 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfwnm\" (UniqueName: \"kubernetes.io/projected/b673b420-6fcb-4021-8ea7-1dce6101f1ad-kube-api-access-pfwnm\") pod \"node-resolver-ngf5j\" (UID: \"b673b420-6fcb-4021-8ea7-1dce6101f1ad\") " pod="openshift-dns/node-resolver-ngf5j" Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.142241 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np8b7\" (UniqueName: \"kubernetes.io/projected/1b816e2d-d250-4c9f-abb2-ff37218ff860-kube-api-access-np8b7\") pod \"multus-additional-cni-plugins-5m2x2\" (UID: \"1b816e2d-d250-4c9f-abb2-ff37218ff860\") " pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.142492 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlktv\" (UniqueName: \"kubernetes.io/projected/0fd62bf0-0229-4bf3-9225-bdfa7ec15be4-kube-api-access-vlktv\") pod \"aws-ebs-csi-driver-node-swmqq\" (UID: \"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.142589 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtt5w\" (UniqueName: \"kubernetes.io/projected/b939e580-7ce8-4e0b-a2c6-6e55f65e25d8-kube-api-access-mtt5w\") pod \"node-ca-ljb4k\" (UID: \"b939e580-7ce8-4e0b-a2c6-6e55f65e25d8\") " pod="openshift-image-registry/node-ca-ljb4k" Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.142743 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8697\" (UniqueName: \"kubernetes.io/projected/5350fb76-01c2-43f7-bd40-29c9b0622283-kube-api-access-v8697\") pod \"tuned-654sp\" (UID: \"5350fb76-01c2-43f7-bd40-29c9b0622283\") " pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.143669 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh2hb\" (UniqueName: \"kubernetes.io/projected/daeb5059-d59d-457e-a5c5-d8545f19b19e-kube-api-access-wh2hb\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:40.187732 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.143919 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8x4r\" (UniqueName: \"kubernetes.io/projected/b8945dd9-218d-4d58-b42e-6c2ca865e382-kube-api-access-d8x4r\") pod \"iptables-alerter-b8c5k\" (UID: \"b8945dd9-218d-4d58-b42e-6c2ca865e382\") " pod="openshift-network-operator/iptables-alerter-b8c5k" Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.200469 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjxsv\" (UniqueName: \"kubernetes.io/projected/9f103464-e957-4cb4-bd09-c0361841a2b5-kube-api-access-gjxsv\") pod \"ovnkube-node-s7stv\" (UID: \"9f103464-e957-4cb4-bd09-c0361841a2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.200475 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lclp9\" (UniqueName: \"kubernetes.io/projected/106e79cf-58a7-4cfd-9738-01fa20d637f1-kube-api-access-lclp9\") pod \"multus-mjcqv\" (UID: \"106e79cf-58a7-4cfd-9738-01fa20d637f1\") " pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:40.202652 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a6046fa193489a163cd70c7bb12503.slice/crio-ccf11901f6a801ed561e60c057df1562148850f9badb6eeff5d32c025ac0e543 WatchSource:0}: Error finding container ccf11901f6a801ed561e60c057df1562148850f9badb6eeff5d32c025ac0e543: Status 404 returned error can't find the container with id ccf11901f6a801ed561e60c057df1562148850f9badb6eeff5d32c025ac0e543 Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:40.202946 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod116152a5548a914d4afdfde51aad9a78.slice/crio-77d635f4ac073893fe5d39b0dd341cc1de868d2b4ccc7912d7f4f95a2bea5709 WatchSource:0}: Error finding container 77d635f4ac073893fe5d39b0dd341cc1de868d2b4ccc7912d7f4f95a2bea5709: Status 404 returned error can't find the container with id 77d635f4ac073893fe5d39b0dd341cc1de868d2b4ccc7912d7f4f95a2bea5709 Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.208363 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.217842 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ljb4k" Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.223316 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mjcqv" Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.228458 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b8c5k" Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:40.233328 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb939e580_7ce8_4e0b_a2c6_6e55f65e25d8.slice/crio-52de0cde0c28489ee44da456d44eeba153bdedf7c513f449f950bdbb47b9cfa2 WatchSource:0}: Error finding container 52de0cde0c28489ee44da456d44eeba153bdedf7c513f449f950bdbb47b9cfa2: Status 404 returned error can't find the container with id 52de0cde0c28489ee44da456d44eeba153bdedf7c513f449f950bdbb47b9cfa2 Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:40.236738 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod106e79cf_58a7_4cfd_9738_01fa20d637f1.slice/crio-8d1c483ddfd23cb1edfdd6c6eac3bfb3cbb6c565a3c8d496b14296fc481830b1 WatchSource:0}: Error finding container 8d1c483ddfd23cb1edfdd6c6eac3bfb3cbb6c565a3c8d496b14296fc481830b1: Status 404 returned error can't find the container with id 8d1c483ddfd23cb1edfdd6c6eac3bfb3cbb6c565a3c8d496b14296fc481830b1 Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:40.243959 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8945dd9_218d_4d58_b42e_6c2ca865e382.slice/crio-27cb6f1ac569e3cfe1d2bd2aa8df682f47b19d1fb27b3574a894f1e2c4975137 WatchSource:0}: Error finding container 27cb6f1ac569e3cfe1d2bd2aa8df682f47b19d1fb27b3574a894f1e2c4975137: Status 404 returned error can't find the container with id 27cb6f1ac569e3cfe1d2bd2aa8df682f47b19d1fb27b3574a894f1e2c4975137 Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.246141 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-45szx" Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:40.253702 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14934d93_f8ca_478d_ba2d_24c70d9ab9ec.slice/crio-a52fc8cffd3a7a0c0158af9fbafa8ce22318e889c9ba812a928304841390cb4a WatchSource:0}: Error finding container a52fc8cffd3a7a0c0158af9fbafa8ce22318e889c9ba812a928304841390cb4a: Status 404 returned error can't find the container with id a52fc8cffd3a7a0c0158af9fbafa8ce22318e889c9ba812a928304841390cb4a Jul 23 17:59:40.300589 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.271845 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5m2x2" Jul 23 17:59:40.414023 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.322560 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" Jul 23 17:59:40.414023 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.322622 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 17:59:40.414023 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:40.331366 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fd62bf0_0229_4bf3_9225_bdfa7ec15be4.slice/crio-49434c9a38c160fcfb87f60ef9972f42cfdd85c81c0b6ac187d776c608fcbe1b WatchSource:0}: Error finding container 49434c9a38c160fcfb87f60ef9972f42cfdd85c81c0b6ac187d776c608fcbe1b: Status 404 returned error can't find the container with id 49434c9a38c160fcfb87f60ef9972f42cfdd85c81c0b6ac187d776c608fcbe1b Jul 23 17:59:40.414023 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:40.331805 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f103464_e957_4cb4_bd09_c0361841a2b5.slice/crio-5088ccba7c9cfa866552e76d08dd7d1b5ba2731e7a2236364b9e7cd0681e6c1b WatchSource:0}: Error finding container 5088ccba7c9cfa866552e76d08dd7d1b5ba2731e7a2236364b9e7cd0681e6c1b: Status 404 returned error can't find the container with id 5088ccba7c9cfa866552e76d08dd7d1b5ba2731e7a2236364b9e7cd0681e6c1b Jul 23 17:59:40.414023 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.364945 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-654sp" Jul 23 17:59:40.414023 ip-10-0-129-187 kubenswrapper[2562]: W0723 17:59:40.373184 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5350fb76_01c2_43f7_bd40_29c9b0622283.slice/crio-b297bc2fe051e442b38a87e5ef0681f0617454911e490c0dd08efecd8e30ae98 WatchSource:0}: Error finding container b297bc2fe051e442b38a87e5ef0681f0617454911e490c0dd08efecd8e30ae98: Status 404 returned error can't find the container with id b297bc2fe051e442b38a87e5ef0681f0617454911e490c0dd08efecd8e30ae98 Jul 23 17:59:40.414023 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.379101 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ngf5j" Jul 23 17:59:40.414023 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.399080 2562 kuberuntime_manager.go:1459] "Unhandled Error" err="container dns-node-resolver start failed in pod node-resolver-ngf5j_openshift-dns(b673b420-6fcb-4021-8ea7-1dce6101f1ad): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 23 17:59:40.414023 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.399109 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-dns/node-resolver-ngf5j" podUID="b673b420-6fcb-4021-8ea7-1dce6101f1ad" Jul 23 17:59:40.536121 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.450674 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:59:40.624516 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.624431 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:40.624516 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.624494 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:40.624736 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.624600 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:40.624736 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.624602 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:40.624736 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.624658 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret podName:822c461b-5e7b-4abb-a090-edacc6b4392c nodeName:}" failed. No retries permitted until 2026-07-23 17:59:41.624644513 +0000 UTC m=+3.112136482 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret") pod "global-pull-secret-syncer-h62kj" (UID: "822c461b-5e7b-4abb-a090-edacc6b4392c") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:40.624736 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.624678 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs podName:daeb5059-d59d-457e-a5c5-d8545f19b19e nodeName:}" failed. No retries permitted until 2026-07-23 17:59:41.62466795 +0000 UTC m=+3.112160115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs") pod "network-metrics-daemon-88l5f" (UID: "daeb5059-d59d-457e-a5c5-d8545f19b19e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:40.725238 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:40.725188 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fhbm\" (UniqueName: \"kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm\") pod \"network-check-target-l8nhz\" (UID: \"4178e19f-ad0f-4366-8f06-471cb052caf5\") " pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:40.725417 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.725376 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:59:40.725417 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.725404 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:59:40.725417 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.725417 2562 projected.go:196] Error preparing data for projected volume kube-api-access-4fhbm for pod openshift-network-diagnostics/network-check-target-l8nhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:40.725572 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:40.725481 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm podName:4178e19f-ad0f-4366-8f06-471cb052caf5 nodeName:}" failed. No retries permitted until 2026-07-23 17:59:41.725461321 +0000 UTC m=+3.212953294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fhbm" (UniqueName: "kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm") pod "network-check-target-l8nhz" (UID: "4178e19f-ad0f-4366-8f06-471cb052caf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:41.024359 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.024278 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:59:41.026849 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.026745 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 17:54:40 +0000 UTC" deadline="2028-02-03 02:47:26.171089167 +0000 UTC" Jul 23 17:59:41.026849 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.026796 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13424h47m45.144296455s" Jul 23 17:59:41.125662 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.125159 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:41.125662 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:41.125306 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 17:59:41.136486 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.136428 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" event={"ID":"9f103464-e957-4cb4-bd09-c0361841a2b5","Type":"ContainerStarted","Data":"5088ccba7c9cfa866552e76d08dd7d1b5ba2731e7a2236364b9e7cd0681e6c1b"} Jul 23 17:59:41.149270 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.149180 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" event={"ID":"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4","Type":"ContainerStarted","Data":"49434c9a38c160fcfb87f60ef9972f42cfdd85c81c0b6ac187d776c608fcbe1b"} Jul 23 17:59:41.172322 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.172189 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b8c5k" event={"ID":"b8945dd9-218d-4d58-b42e-6c2ca865e382","Type":"ContainerStarted","Data":"27cb6f1ac569e3cfe1d2bd2aa8df682f47b19d1fb27b3574a894f1e2c4975137"} Jul 23 17:59:41.191779 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.191678 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ljb4k" event={"ID":"b939e580-7ce8-4e0b-a2c6-6e55f65e25d8","Type":"ContainerStarted","Data":"52de0cde0c28489ee44da456d44eeba153bdedf7c513f449f950bdbb47b9cfa2"} Jul 23 17:59:41.210044 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.209976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-187.ec2.internal" event={"ID":"12a6046fa193489a163cd70c7bb12503","Type":"ContainerStarted","Data":"ccf11901f6a801ed561e60c057df1562148850f9badb6eeff5d32c025ac0e543"} Jul 23 17:59:41.226871 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.226277 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" event={"ID":"116152a5548a914d4afdfde51aad9a78","Type":"ContainerStarted","Data":"77d635f4ac073893fe5d39b0dd341cc1de868d2b4ccc7912d7f4f95a2bea5709"} Jul 23 17:59:41.243476 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.243439 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ngf5j" event={"ID":"b673b420-6fcb-4021-8ea7-1dce6101f1ad","Type":"ContainerStarted","Data":"0063e418b7fad8cbb0df96e02905f91a48ce2941ab5c8386354019e1362b6d52"} Jul 23 17:59:41.247059 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:41.247022 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e83687a3e51e78c37199d4d4504faf8bfeec5d995d123db950ef57fe67957539\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-dns/node-resolver-ngf5j" podUID="b673b420-6fcb-4021-8ea7-1dce6101f1ad" Jul 23 17:59:41.250504 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.250430 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-654sp" event={"ID":"5350fb76-01c2-43f7-bd40-29c9b0622283","Type":"ContainerStarted","Data":"b297bc2fe051e442b38a87e5ef0681f0617454911e490c0dd08efecd8e30ae98"} Jul 23 17:59:41.273948 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.273902 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5m2x2" event={"ID":"1b816e2d-d250-4c9f-abb2-ff37218ff860","Type":"ContainerStarted","Data":"a0b320a750aa47d73dd6e3686f959882aaf7b8269e860f2a5f5049872b93b0f1"} Jul 23 17:59:41.291157 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.290965 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-45szx" event={"ID":"14934d93-f8ca-478d-ba2d-24c70d9ab9ec","Type":"ContainerStarted","Data":"a52fc8cffd3a7a0c0158af9fbafa8ce22318e889c9ba812a928304841390cb4a"} Jul 23 17:59:41.314488 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.314455 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mjcqv" event={"ID":"106e79cf-58a7-4cfd-9738-01fa20d637f1","Type":"ContainerStarted","Data":"8d1c483ddfd23cb1edfdd6c6eac3bfb3cbb6c565a3c8d496b14296fc481830b1"} Jul 23 17:59:41.632273 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.632063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:41.632273 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.632145 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:41.632501 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:41.632281 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:41.632501 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:41.632337 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs podName:daeb5059-d59d-457e-a5c5-d8545f19b19e nodeName:}" failed. No retries permitted until 2026-07-23 17:59:43.63231968 +0000 UTC m=+5.119811657 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs") pod "network-metrics-daemon-88l5f" (UID: "daeb5059-d59d-457e-a5c5-d8545f19b19e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:41.633517 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:41.632719 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:41.633517 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:41.632780 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret podName:822c461b-5e7b-4abb-a090-edacc6b4392c nodeName:}" failed. No retries permitted until 2026-07-23 17:59:43.632756869 +0000 UTC m=+5.120248842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret") pod "global-pull-secret-syncer-h62kj" (UID: "822c461b-5e7b-4abb-a090-edacc6b4392c") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:41.732475 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.732434 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fhbm\" (UniqueName: \"kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm\") pod \"network-check-target-l8nhz\" (UID: \"4178e19f-ad0f-4366-8f06-471cb052caf5\") " pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:41.732656 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:41.732611 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:59:41.732656 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:41.732632 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:59:41.732656 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:41.732643 2562 projected.go:196] Error preparing data for projected volume kube-api-access-4fhbm for pod openshift-network-diagnostics/network-check-target-l8nhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:41.732797 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:41.732712 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm podName:4178e19f-ad0f-4366-8f06-471cb052caf5 nodeName:}" failed. No retries permitted until 2026-07-23 17:59:43.732692258 +0000 UTC m=+5.220184243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fhbm" (UniqueName: "kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm") pod "network-check-target-l8nhz" (UID: "4178e19f-ad0f-4366-8f06-471cb052caf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:41.860339 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:41.860027 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:59:42.028057 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:42.027935 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 17:54:40 +0000 UTC" deadline="2028-01-24 23:00:00.06814073 +0000 UTC" Jul 23 17:59:42.028057 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:42.027979 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13205h0m18.040167614s" Jul 23 17:59:42.124927 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:42.124895 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:42.125106 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:42.125034 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 17:59:42.125469 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:42.125447 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:42.125577 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:42.125557 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 17:59:42.885062 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:42.884794 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:59:43.127110 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:43.126056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:43.127110 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:43.126192 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 17:59:43.645954 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:43.645868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:43.646135 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:43.645957 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:43.646135 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:43.646087 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:43.646281 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:43.646155 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs podName:daeb5059-d59d-457e-a5c5-d8545f19b19e nodeName:}" failed. No retries permitted until 2026-07-23 17:59:47.646134457 +0000 UTC m=+9.133626438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs") pod "network-metrics-daemon-88l5f" (UID: "daeb5059-d59d-457e-a5c5-d8545f19b19e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:43.646600 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:43.646580 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:43.646697 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:43.646637 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret podName:822c461b-5e7b-4abb-a090-edacc6b4392c nodeName:}" failed. No retries permitted until 2026-07-23 17:59:47.646621847 +0000 UTC m=+9.134113817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret") pod "global-pull-secret-syncer-h62kj" (UID: "822c461b-5e7b-4abb-a090-edacc6b4392c") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:43.747176 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:43.747123 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fhbm\" (UniqueName: \"kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm\") pod \"network-check-target-l8nhz\" (UID: \"4178e19f-ad0f-4366-8f06-471cb052caf5\") " pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:43.747371 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:43.747332 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:59:43.747371 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:43.747362 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:59:43.747484 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:43.747377 2562 projected.go:196] Error preparing data for projected volume kube-api-access-4fhbm for pod openshift-network-diagnostics/network-check-target-l8nhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:43.747484 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:43.747444 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm podName:4178e19f-ad0f-4366-8f06-471cb052caf5 nodeName:}" failed. No retries permitted until 2026-07-23 17:59:47.747421636 +0000 UTC m=+9.234913625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fhbm" (UniqueName: "kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm") pod "network-check-target-l8nhz" (UID: "4178e19f-ad0f-4366-8f06-471cb052caf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:44.124958 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:44.124862 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:44.125115 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:44.124999 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 17:59:44.125115 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:44.125046 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:44.125235 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:44.125168 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 17:59:45.126879 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:45.126847 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:45.127358 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:45.126990 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 17:59:46.125064 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:46.124917 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:46.125248 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:46.125066 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 17:59:46.125248 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:46.125195 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:46.125378 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:46.125320 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 17:59:47.124796 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:47.124733 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:47.125228 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:47.124876 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 17:59:47.680424 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:47.680206 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:47.680424 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:47.680293 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:47.680424 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:47.680395 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:47.680757 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:47.680483 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs podName:daeb5059-d59d-457e-a5c5-d8545f19b19e nodeName:}" failed. No retries permitted until 2026-07-23 17:59:55.680458263 +0000 UTC m=+17.167950255 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs") pod "network-metrics-daemon-88l5f" (UID: "daeb5059-d59d-457e-a5c5-d8545f19b19e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:47.680757 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:47.680402 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:47.680757 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:47.680573 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret podName:822c461b-5e7b-4abb-a090-edacc6b4392c nodeName:}" failed. No retries permitted until 2026-07-23 17:59:55.680555992 +0000 UTC m=+17.168047967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret") pod "global-pull-secret-syncer-h62kj" (UID: "822c461b-5e7b-4abb-a090-edacc6b4392c") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:47.781449 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:47.781410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fhbm\" (UniqueName: \"kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm\") pod \"network-check-target-l8nhz\" (UID: \"4178e19f-ad0f-4366-8f06-471cb052caf5\") " pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:47.781626 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:47.781606 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:59:47.781690 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:47.781631 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:59:47.781690 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:47.781644 2562 projected.go:196] Error preparing data for projected volume kube-api-access-4fhbm for pod openshift-network-diagnostics/network-check-target-l8nhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:47.781778 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:47.781712 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm podName:4178e19f-ad0f-4366-8f06-471cb052caf5 nodeName:}" failed. No retries permitted until 2026-07-23 17:59:55.78169221 +0000 UTC m=+17.269184182 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fhbm" (UniqueName: "kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm") pod "network-check-target-l8nhz" (UID: "4178e19f-ad0f-4366-8f06-471cb052caf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:48.124657 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:48.124561 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:48.124824 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:48.124578 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:48.124824 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:48.124702 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 17:59:48.124824 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:48.124754 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 17:59:49.125787 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:49.125676 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:49.126225 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:49.125814 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 17:59:50.124659 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:50.124613 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:50.124659 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:50.124639 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:50.124906 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:50.124741 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 17:59:50.124906 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:50.124858 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 17:59:51.124702 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:51.124659 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:51.125120 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:51.124802 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 17:59:52.125428 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:52.125396 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:52.125428 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:52.125414 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:52.125946 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:52.125517 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 17:59:52.125946 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:52.125669 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 17:59:53.124784 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:53.124751 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:53.124962 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:53.124872 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 17:59:54.124977 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:54.124935 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:54.125474 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:54.124937 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:54.125474 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:54.125066 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 17:59:54.125474 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:54.125142 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 17:59:55.124679 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:55.124642 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:55.124836 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:55.124778 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 17:59:55.741927 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:55.741883 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:55.742369 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:55.741957 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:55.742369 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:55.742027 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:55.742369 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:55.742039 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:55.742369 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:55.742097 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs podName:daeb5059-d59d-457e-a5c5-d8545f19b19e nodeName:}" failed. No retries permitted until 2026-07-23 18:00:11.742075846 +0000 UTC m=+33.229568030 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs") pod "network-metrics-daemon-88l5f" (UID: "daeb5059-d59d-457e-a5c5-d8545f19b19e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:59:55.742369 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:55.742117 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret podName:822c461b-5e7b-4abb-a090-edacc6b4392c nodeName:}" failed. No retries permitted until 2026-07-23 18:00:11.742107043 +0000 UTC m=+33.229599013 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret") pod "global-pull-secret-syncer-h62kj" (UID: "822c461b-5e7b-4abb-a090-edacc6b4392c") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:59:55.842566 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:55.842532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fhbm\" (UniqueName: \"kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm\") pod \"network-check-target-l8nhz\" (UID: \"4178e19f-ad0f-4366-8f06-471cb052caf5\") " pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:55.842771 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:55.842718 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:59:55.842771 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:55.842756 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:59:55.842771 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:55.842771 2562 projected.go:196] Error preparing data for projected volume kube-api-access-4fhbm for pod openshift-network-diagnostics/network-check-target-l8nhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:55.842934 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:55.842845 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm podName:4178e19f-ad0f-4366-8f06-471cb052caf5 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:11.842830143 +0000 UTC m=+33.330322116 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fhbm" (UniqueName: "kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm") pod "network-check-target-l8nhz" (UID: "4178e19f-ad0f-4366-8f06-471cb052caf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:59:56.124573 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:56.124482 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:56.124573 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:56.124520 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:56.124835 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:56.124796 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 17:59:56.124954 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:56.124905 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 17:59:57.125451 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:57.125417 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:57.125819 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:57.125535 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 17:59:58.124833 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:58.124796 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 17:59:58.125037 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:58.124848 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 17:59:58.125037 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:58.124936 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 17:59:58.125158 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:58.125076 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 17:59:59.125324 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:59.125143 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 17:59:59.125659 ip-10-0-129-187 kubenswrapper[2562]: E0723 17:59:59.125433 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 17:59:59.360928 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:59.360892 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mjcqv" event={"ID":"106e79cf-58a7-4cfd-9738-01fa20d637f1","Type":"ContainerStarted","Data":"8701cb2bbef2831fc9f131c72ea9491cb138107df4f9fee89bbf61a18985965b"} Jul 23 17:59:59.372927 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:59.372889 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" event={"ID":"9f103464-e957-4cb4-bd09-c0361841a2b5","Type":"ContainerStarted","Data":"28d6863d1d02b3850cf29ecbb78c0bb52a786be602b95d16a5ed17e9414df699"} Jul 23 17:59:59.379261 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:59.379159 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-187.ec2.internal" event={"ID":"12a6046fa193489a163cd70c7bb12503","Type":"ContainerStarted","Data":"1886a1e47449f92341a9f86b27ada47a41e15c235c5b41296f0d078b9c6b669a"} Jul 23 17:59:59.384588 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:59.384530 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mjcqv" podStartSLOduration=1.363789248 podStartE2EDuration="20.384512527s" totalImagesPullingTime="19.020723279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:59:59.383830919 +0000 UTC m=+20.871322910" watchObservedRunningTime="2026-07-23 17:59:59.384512527 +0000 UTC m=+20.872004520" Jul 23 17:59:59.387595 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:59.387565 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-654sp" event={"ID":"5350fb76-01c2-43f7-bd40-29c9b0622283","Type":"ContainerStarted","Data":"9a80916c443384229e453d47ced4d59479e483ce9d1570314557479131567938"} Jul 23 17:59:59.399705 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:59.399656 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-187.ec2.internal" podStartSLOduration=20.399636852 podStartE2EDuration="20.399636852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:59:59.399070805 +0000 UTC m=+20.886562831" watchObservedRunningTime="2026-07-23 17:59:59.399636852 +0000 UTC m=+20.887128845" Jul 23 17:59:59.419993 ip-10-0-129-187 kubenswrapper[2562]: I0723 17:59:59.419949 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-654sp" podStartSLOduration=1.709314056 podStartE2EDuration="20.419933866s" totalImagesPullingTime="18.71061981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:59:59.419666588 +0000 UTC m=+20.907158582" watchObservedRunningTime="2026-07-23 17:59:59.419933866 +0000 UTC m=+20.907425855" Jul 23 18:00:00.124545 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.124514 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:00.124714 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.124514 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:00.124714 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:00.124651 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 18:00:00.124838 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:00.124713 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 18:00:00.390282 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.390243 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ngf5j" event={"ID":"b673b420-6fcb-4021-8ea7-1dce6101f1ad","Type":"ContainerStarted","Data":"30ebd4a33dd4287f44fe334eefdb4f4a4e0ad9861a721d697cacf18765c0d0ed"} Jul 23 18:00:00.391706 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.391684 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b816e2d-d250-4c9f-abb2-ff37218ff860" containerID="d55cb20eaf3c20c2272ed9bdd2eb8710b5eed8e9ecc61575d5eacb359acc7137" exitCode=0 Jul 23 18:00:00.391798 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.391750 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5m2x2" event={"ID":"1b816e2d-d250-4c9f-abb2-ff37218ff860","Type":"ContainerDied","Data":"d55cb20eaf3c20c2272ed9bdd2eb8710b5eed8e9ecc61575d5eacb359acc7137"} Jul 23 18:00:00.394920 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.394892 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-45szx" event={"ID":"14934d93-f8ca-478d-ba2d-24c70d9ab9ec","Type":"ContainerStarted","Data":"40c1094fe330d3c599c9a156c7b475359eaee0071503ffda594a59a5b12470da"} Jul 23 18:00:00.398024 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.398003 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" event={"ID":"9f103464-e957-4cb4-bd09-c0361841a2b5","Type":"ContainerStarted","Data":"e626ab889ffd6e9f0e2fc1e80f8f1f6550e2c6c5be073426acaf25a9bc45c19a"} Jul 23 18:00:00.398118 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.398063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" event={"ID":"9f103464-e957-4cb4-bd09-c0361841a2b5","Type":"ContainerStarted","Data":"13bc06eefb0b409caddf2f8e25815f9205a203b512d608550206f1413d54006f"} Jul 23 18:00:00.400910 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.398442 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" event={"ID":"9f103464-e957-4cb4-bd09-c0361841a2b5","Type":"ContainerStarted","Data":"4ac81675ec28902ba4db4501f192d382e88e7881a4ddd9758e03b3489a18a5c2"} Jul 23 18:00:00.400910 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.398536 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" event={"ID":"9f103464-e957-4cb4-bd09-c0361841a2b5","Type":"ContainerStarted","Data":"1e5fa1b00d7fbcc53a20914edaa254986a451b318674ddf733e7d3785a88db96"} Jul 23 18:00:00.400910 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.399161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" event={"ID":"9f103464-e957-4cb4-bd09-c0361841a2b5","Type":"ContainerStarted","Data":"c0a92537745a67476289ce929a9212ebea24328ddfeed11c6b4367e9693fb737"} Jul 23 18:00:00.402260 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.402239 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" event={"ID":"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4","Type":"ContainerStarted","Data":"c9665a82bec9064e63bfac2e2c0c653471d356d2b20d8ef2c19b0d88e5477ede"} Jul 23 18:00:00.403556 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.403536 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ljb4k" event={"ID":"b939e580-7ce8-4e0b-a2c6-6e55f65e25d8","Type":"ContainerStarted","Data":"87a34f927e7130720f5f58526cf69a01c8c9b0d44b1daa3c81aeed8f2652fb70"} Jul 23 18:00:00.404841 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.404820 2562 generic.go:356] "Generic (PLEG): container finished" podID="116152a5548a914d4afdfde51aad9a78" containerID="66df32fc155b40145ce91f252f9d32fe35f69223a48b164fe924ea43a9ba36ea" exitCode=0 Jul 23 18:00:00.404906 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.404887 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" event={"ID":"116152a5548a914d4afdfde51aad9a78","Type":"ContainerDied","Data":"66df32fc155b40145ce91f252f9d32fe35f69223a48b164fe924ea43a9ba36ea"} Jul 23 18:00:00.409167 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.409129 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ngf5j" podStartSLOduration=21.409116267 podStartE2EDuration="21.409116267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-23 18:00:00.407832056 +0000 UTC m=+21.895324046" watchObservedRunningTime="2026-07-23 18:00:00.409116267 +0000 UTC m=+21.896608258" Jul 23 18:00:00.460240 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.460172 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ljb4k" podStartSLOduration=2.771748782 podStartE2EDuration="21.460152765s" totalImagesPullingTime="18.688403983s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:00.460061282 +0000 UTC m=+21.947553273" watchObservedRunningTime="2026-07-23 18:00:00.460152765 +0000 UTC m=+21.947644756" Jul 23 18:00:00.476702 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.476496 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-45szx" podStartSLOduration=7.172068951 podStartE2EDuration="21.476476438s" totalImagesPullingTime="14.304407487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:00.476186974 +0000 UTC m=+21.963678965" watchObservedRunningTime="2026-07-23 18:00:00.476476438 +0000 UTC m=+21.963968430" Jul 23 18:00:00.570616 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.570585 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:00.648853 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.648823 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:00.684052 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.684030 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 18:00:00.958618 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.958573 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:00.970574 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.970469 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T18:00:00.684045374Z","UUID":"5e333b8e-63c8-4056-b7f5-6952bdcf8f33","Handler":null,"Name":"","Endpoint":""} Jul 23 18:00:00.972168 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.972145 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 18:00:00.972300 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:00.972194 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 18:00:01.057112 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:01.057077 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:01.125457 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:01.125429 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 18:00:01.125621 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:01.125559 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 18:00:01.185457 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:01.185423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:01.384674 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:01.384579 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:01.409610 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:01.409538 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" event={"ID":"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4","Type":"ContainerStarted","Data":"cc8ad814370fd5333c173eeb02e2602f9c0ea84f3d1c3b8381b963bf1d24f4ff"} Jul 23 18:00:01.411302 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:01.411272 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b8c5k" event={"ID":"b8945dd9-218d-4d58-b42e-6c2ca865e382","Type":"ContainerStarted","Data":"b033c5810b0e8614c1d5bae3a6a3c3a934d76f8712b52cb98c66ca819e8d6cf3"} Jul 23 18:00:01.413377 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:01.413352 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" event={"ID":"116152a5548a914d4afdfde51aad9a78","Type":"ContainerStarted","Data":"00a27485dc129b695839997a9f9b7d2a7d77270245707e17bd9c0606e09c3b49"} Jul 23 18:00:01.428964 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:01.428903 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b8c5k" podStartSLOduration=3.883323087 podStartE2EDuration="22.428881981s" totalImagesPullingTime="18.545558894s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:01.42824199 +0000 UTC m=+22.915733976" watchObservedRunningTime="2026-07-23 18:00:01.428881981 +0000 UTC m=+22.916373973" Jul 23 18:00:01.450001 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:01.449951 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-187.ec2.internal" podStartSLOduration=22.449933133 podStartE2EDuration="22.449933133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:01.449812385 +0000 UTC m=+22.937304376" watchObservedRunningTime="2026-07-23 18:00:01.449933133 +0000 UTC m=+22.937425125" Jul 23 18:00:01.590351 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:01.590317 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:01.862296 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:01.862240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:02.124475 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:02.124450 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:02.124603 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:02.124581 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 18:00:02.124667 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:02.124636 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:02.124790 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:02.124763 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 18:00:02.419783 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:02.419688 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" event={"ID":"9f103464-e957-4cb4-bd09-c0361841a2b5","Type":"ContainerStarted","Data":"f617ff00b44ca04b585ddd1c082ff3f902ef8b98ce38797449d00d4dab6cada1"} Jul 23 18:00:02.717544 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:02.717509 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-45szx" Jul 23 18:00:02.718241 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:02.718173 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-45szx" Jul 23 18:00:03.125307 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:03.125207 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 18:00:03.125475 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:03.125361 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 18:00:03.424456 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:03.424380 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" event={"ID":"0fd62bf0-0229-4bf3-9225-bdfa7ec15be4","Type":"ContainerStarted","Data":"fecc42cb1392e771405d3b494f2daa7cd51e1f2572908b7b7e9f819359feb6e3"} Jul 23 18:00:03.445905 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:03.445849 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-swmqq" podStartSLOduration=3.201111212 podStartE2EDuration="24.445832864s" totalImagesPullingTime="21.244721652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:03.445800389 +0000 UTC m=+24.933292382" watchObservedRunningTime="2026-07-23 18:00:03.445832864 +0000 UTC m=+24.933324833" Jul 23 18:00:04.124801 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:04.124760 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:04.124985 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:04.124805 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:04.124985 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:04.124914 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 18:00:04.125085 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:04.125035 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 18:00:05.124943 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:05.124758 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 18:00:05.125434 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:05.125027 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 18:00:05.429165 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:05.429081 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b816e2d-d250-4c9f-abb2-ff37218ff860" containerID="124fad135def28da063a89d5120f05515270d94e68bd75cbfd73441d9d0fea6e" exitCode=0 Jul 23 18:00:05.429360 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:05.429168 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5m2x2" event={"ID":"1b816e2d-d250-4c9f-abb2-ff37218ff860","Type":"ContainerDied","Data":"124fad135def28da063a89d5120f05515270d94e68bd75cbfd73441d9d0fea6e"} Jul 23 18:00:05.432851 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:05.432819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" event={"ID":"9f103464-e957-4cb4-bd09-c0361841a2b5","Type":"ContainerStarted","Data":"8332d215e3d4b159c0335508eab2a6cc7861b4c15128b9705cf9426b0da32a63"} Jul 23 18:00:05.433087 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:05.433060 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 18:00:05.448048 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:05.448026 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 18:00:05.536970 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:05.536923 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" podStartSLOduration=7.740915145 podStartE2EDuration="26.536908919s" totalImagesPullingTime="18.795993774s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:05.536549029 +0000 UTC m=+27.024041019" watchObservedRunningTime="2026-07-23 18:00:05.536908919 +0000 UTC m=+27.024400910" Jul 23 18:00:06.124760 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.124726 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:06.124973 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.124742 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:06.124973 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:06.124869 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 18:00:06.124973 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:06.124963 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 18:00:06.410325 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.410239 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l8nhz"] Jul 23 18:00:06.415890 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.415863 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h62kj"] Jul 23 18:00:06.416011 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.415988 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 18:00:06.416126 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:06.416100 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 18:00:06.416612 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.416586 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-88l5f"] Jul 23 18:00:06.434666 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.434642 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:06.434805 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.434642 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:06.434805 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:06.434783 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 18:00:06.434927 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:06.434842 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 18:00:06.434927 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.434860 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 18:00:06.435258 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.435242 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 18:00:06.450986 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.450961 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 18:00:06.841887 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.841867 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-45szx" Jul 23 18:00:06.842011 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.842000 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 23 18:00:06.842919 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.842552 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-45szx" Jul 23 18:00:06.994838 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:06.994639 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:07.438322 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:07.438291 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b816e2d-d250-4c9f-abb2-ff37218ff860" containerID="63bf88f4ff398e2b0078f8b92d74e952c6561f073f98431bdd2dfeaff17e4bfd" exitCode=0 Jul 23 18:00:07.438980 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:07.438388 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5m2x2" event={"ID":"1b816e2d-d250-4c9f-abb2-ff37218ff860","Type":"ContainerDied","Data":"63bf88f4ff398e2b0078f8b92d74e952c6561f073f98431bdd2dfeaff17e4bfd"} Jul 23 18:00:08.125341 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:08.125315 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 18:00:08.125512 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:08.125320 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:08.125512 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:08.125399 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 18:00:08.125512 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:08.125410 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:08.125512 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:08.125495 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 18:00:08.125647 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:08.125573 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 18:00:09.448420 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:09.448391 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b816e2d-d250-4c9f-abb2-ff37218ff860" containerID="659917f4b0d0146624e080982fb55af082fae708ffdf4dd4672c984d9e9a0c05" exitCode=0 Jul 23 18:00:09.448830 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:09.448436 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5m2x2" event={"ID":"1b816e2d-d250-4c9f-abb2-ff37218ff860","Type":"ContainerDied","Data":"659917f4b0d0146624e080982fb55af082fae708ffdf4dd4672c984d9e9a0c05"} Jul 23 18:00:10.125160 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:10.125120 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 18:00:10.125377 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:10.125122 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:10.125377 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:10.125283 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h62kj" podUID="822c461b-5e7b-4abb-a090-edacc6b4392c" Jul 23 18:00:10.125377 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:10.125327 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:10.125502 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:10.125424 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88l5f" podUID="daeb5059-d59d-457e-a5c5-d8545f19b19e" Jul 23 18:00:10.125541 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:10.125505 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l8nhz" podUID="4178e19f-ad0f-4366-8f06-471cb052caf5" Jul 23 18:00:11.765411 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.765365 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:11.765781 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.765428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 18:00:11.765781 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:11.765524 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:00:11.765781 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:11.765557 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:00:11.765781 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:11.765595 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs podName:daeb5059-d59d-457e-a5c5-d8545f19b19e nodeName:}" failed. No retries permitted until 2026-07-23 18:00:43.765570351 +0000 UTC m=+65.253062324 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs") pod "network-metrics-daemon-88l5f" (UID: "daeb5059-d59d-457e-a5c5-d8545f19b19e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:00:11.765781 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:11.765611 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret podName:822c461b-5e7b-4abb-a090-edacc6b4392c nodeName:}" failed. No retries permitted until 2026-07-23 18:00:43.765604652 +0000 UTC m=+65.253096625 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret") pod "global-pull-secret-syncer-h62kj" (UID: "822c461b-5e7b-4abb-a090-edacc6b4392c") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:00:11.784876 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.784845 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-187.ec2.internal" event="NodeReady" Jul 23 18:00:11.785026 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.784991 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 18:00:11.840011 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.839976 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hzc9d"] Jul 23 18:00:11.842222 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.842195 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:11.842777 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.842725 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fsxnc"] Jul 23 18:00:11.844680 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.844661 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fsxnc" Jul 23 18:00:11.845510 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.845490 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lb24m\"" Jul 23 18:00:11.845644 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.845617 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 18:00:11.845937 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.845919 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 18:00:11.847092 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.847063 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k2gdw\"" Jul 23 18:00:11.847190 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.847101 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 18:00:11.847337 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.847321 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 18:00:11.847956 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.847938 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 18:00:11.859199 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.858995 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hzc9d"] Jul 23 18:00:11.859658 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.859604 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fsxnc"] Jul 23 18:00:11.866462 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.866440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fhbm\" (UniqueName: \"kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm\") pod \"network-check-target-l8nhz\" (UID: \"4178e19f-ad0f-4366-8f06-471cb052caf5\") " pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:11.866571 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:11.866562 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:00:11.866628 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:11.866581 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:00:11.866628 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:11.866594 2562 projected.go:196] Error preparing data for projected volume kube-api-access-4fhbm for pod openshift-network-diagnostics/network-check-target-l8nhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:00:11.866971 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:11.866951 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm podName:4178e19f-ad0f-4366-8f06-471cb052caf5 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:43.866912316 +0000 UTC m=+65.354404294 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fhbm" (UniqueName: "kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm") pod "network-check-target-l8nhz" (UID: "4178e19f-ad0f-4366-8f06-471cb052caf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:00:11.879995 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.879964 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:11.966959 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.966930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:11.966959 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.966959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0833f914-e35c-4a6d-8a60-0e99269bee04-cert\") pod \"ingress-canary-fsxnc\" (UID: \"0833f914-e35c-4a6d-8a60-0e99269bee04\") " pod="openshift-ingress-canary/ingress-canary-fsxnc" Jul 23 18:00:11.967201 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.966981 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbdhd\" (UniqueName: \"kubernetes.io/projected/0833f914-e35c-4a6d-8a60-0e99269bee04-kube-api-access-cbdhd\") pod \"ingress-canary-fsxnc\" (UID: \"0833f914-e35c-4a6d-8a60-0e99269bee04\") " pod="openshift-ingress-canary/ingress-canary-fsxnc" Jul 23 18:00:11.967201 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.966999 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-tmp-dir\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:11.967201 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.967098 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-config-volume\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:11.967201 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:11.967126 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2s6s\" (UniqueName: \"kubernetes.io/projected/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-kube-api-access-m2s6s\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:12.045653 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.045617 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:12.067812 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.067761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:12.068007 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.067826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0833f914-e35c-4a6d-8a60-0e99269bee04-cert\") pod \"ingress-canary-fsxnc\" (UID: \"0833f914-e35c-4a6d-8a60-0e99269bee04\") " pod="openshift-ingress-canary/ingress-canary-fsxnc" Jul 23 18:00:12.068007 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.067853 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbdhd\" (UniqueName: \"kubernetes.io/projected/0833f914-e35c-4a6d-8a60-0e99269bee04-kube-api-access-cbdhd\") pod \"ingress-canary-fsxnc\" (UID: \"0833f914-e35c-4a6d-8a60-0e99269bee04\") " pod="openshift-ingress-canary/ingress-canary-fsxnc" Jul 23 18:00:12.068007 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.067880 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-tmp-dir\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:12.068007 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:12.067889 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:00:12.068007 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:12.067958 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls podName:84ca6f4e-056f-4d1f-93a4-2e910ef06f08 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:12.567939982 +0000 UTC m=+34.055431965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls") pod "dns-default-hzc9d" (UID: "84ca6f4e-056f-4d1f-93a4-2e910ef06f08") : secret "dns-default-metrics-tls" not found Jul 23 18:00:12.068225 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.068001 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-config-volume\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:12.068225 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.068042 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2s6s\" (UniqueName: \"kubernetes.io/projected/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-kube-api-access-m2s6s\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:12.068309 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.068245 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-tmp-dir\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:12.068737 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.068700 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-config-volume\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:12.072860 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.072839 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0833f914-e35c-4a6d-8a60-0e99269bee04-cert\") pod \"ingress-canary-fsxnc\" (UID: \"0833f914-e35c-4a6d-8a60-0e99269bee04\") " pod="openshift-ingress-canary/ingress-canary-fsxnc" Jul 23 18:00:12.078892 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.078863 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2s6s\" (UniqueName: \"kubernetes.io/projected/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-kube-api-access-m2s6s\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:12.079008 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.078993 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbdhd\" (UniqueName: \"kubernetes.io/projected/0833f914-e35c-4a6d-8a60-0e99269bee04-kube-api-access-cbdhd\") pod \"ingress-canary-fsxnc\" (UID: \"0833f914-e35c-4a6d-8a60-0e99269bee04\") " pod="openshift-ingress-canary/ingress-canary-fsxnc" Jul 23 18:00:12.124606 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.124557 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:12.124606 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.124590 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 18:00:12.124839 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.124557 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:12.127833 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.127810 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 18:00:12.127976 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.127935 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hv4kc\"" Jul 23 18:00:12.128372 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.128350 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 18:00:12.128919 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.128889 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 18:00:12.129035 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.128988 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zbh2j\"" Jul 23 18:00:12.134910 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.134886 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 18:00:12.147583 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.147504 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:12.160814 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.160786 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fsxnc" Jul 23 18:00:12.566000 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.565820 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-9qgsr"] Jul 23 18:00:12.568085 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.568065 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.570739 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.570712 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 23 18:00:12.570868 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.570719 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-cdq99\"" Jul 23 18:00:12.570868 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.570724 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 23 18:00:12.571487 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.571464 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 23 18:00:12.571851 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.571830 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:12.571987 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:12.571956 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:00:12.572105 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:12.572022 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls podName:84ca6f4e-056f-4d1f-93a4-2e910ef06f08 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:13.572001749 +0000 UTC m=+35.059493734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls") pod "dns-default-hzc9d" (UID: "84ca6f4e-056f-4d1f-93a4-2e910ef06f08") : secret "dns-default-metrics-tls" not found Jul 23 18:00:12.574180 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.574157 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 23 18:00:12.576124 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.576014 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 23 18:00:12.584572 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.584548 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-9qgsr"] Jul 23 18:00:12.673182 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.673148 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff788e6b-0522-4742-8f8f-d100924612a0-serving-cert\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.673382 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.673223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qh9z\" (UniqueName: \"kubernetes.io/projected/ff788e6b-0522-4742-8f8f-d100924612a0-kube-api-access-5qh9z\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.673382 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.673301 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff788e6b-0522-4742-8f8f-d100924612a0-config\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.673382 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.673324 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ff788e6b-0522-4742-8f8f-d100924612a0-trusted-ca\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.774225 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.774174 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff788e6b-0522-4742-8f8f-d100924612a0-config\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.774225 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.774232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ff788e6b-0522-4742-8f8f-d100924612a0-trusted-ca\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.774735 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.774285 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff788e6b-0522-4742-8f8f-d100924612a0-serving-cert\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.774735 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.774311 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qh9z\" (UniqueName: \"kubernetes.io/projected/ff788e6b-0522-4742-8f8f-d100924612a0-kube-api-access-5qh9z\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.774924 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.774900 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff788e6b-0522-4742-8f8f-d100924612a0-config\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.775617 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.775589 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ff788e6b-0522-4742-8f8f-d100924612a0-trusted-ca\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.776780 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.776761 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff788e6b-0522-4742-8f8f-d100924612a0-serving-cert\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.783682 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.783658 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qh9z\" (UniqueName: \"kubernetes.io/projected/ff788e6b-0522-4742-8f8f-d100924612a0-kube-api-access-5qh9z\") pod \"console-operator-565b7b569d-9qgsr\" (UID: \"ff788e6b-0522-4742-8f8f-d100924612a0\") " pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:12.878910 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:12.878826 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:13.573354 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:13.573324 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-9qgsr"] Jul 23 18:00:13.576085 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:13.576053 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fsxnc"] Jul 23 18:00:13.578888 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:13.578860 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff788e6b_0522_4742_8f8f_d100924612a0.slice/crio-a1a0c865c83c66dd6980a361d3dea74934d302e7a997ff4752dab3328457fab7 WatchSource:0}: Error finding container a1a0c865c83c66dd6980a361d3dea74934d302e7a997ff4752dab3328457fab7: Status 404 returned error can't find the container with id a1a0c865c83c66dd6980a361d3dea74934d302e7a997ff4752dab3328457fab7 Jul 23 18:00:13.579519 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:13.579503 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0833f914_e35c_4a6d_8a60_0e99269bee04.slice/crio-dca57b7b0efbbcc1d9c6749279ed5f483492523e2ecad62954d3b9130d78ca29 WatchSource:0}: Error finding container dca57b7b0efbbcc1d9c6749279ed5f483492523e2ecad62954d3b9130d78ca29: Status 404 returned error can't find the container with id dca57b7b0efbbcc1d9c6749279ed5f483492523e2ecad62954d3b9130d78ca29 Jul 23 18:00:13.581871 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:13.581848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:13.582000 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:13.581967 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:00:13.582055 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:13.582039 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls podName:84ca6f4e-056f-4d1f-93a4-2e910ef06f08 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:15.582020038 +0000 UTC m=+37.069512008 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls") pod "dns-default-hzc9d" (UID: "84ca6f4e-056f-4d1f-93a4-2e910ef06f08") : secret "dns-default-metrics-tls" not found Jul 23 18:00:14.464001 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:14.463960 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" event={"ID":"ff788e6b-0522-4742-8f8f-d100924612a0","Type":"ContainerStarted","Data":"a1a0c865c83c66dd6980a361d3dea74934d302e7a997ff4752dab3328457fab7"} Jul 23 18:00:14.465339 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:14.465315 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fsxnc" event={"ID":"0833f914-e35c-4a6d-8a60-0e99269bee04","Type":"ContainerStarted","Data":"dca57b7b0efbbcc1d9c6749279ed5f483492523e2ecad62954d3b9130d78ca29"} Jul 23 18:00:14.468318 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:14.468277 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b816e2d-d250-4c9f-abb2-ff37218ff860" containerID="dc5e9cebf5a7e3df0282aef20817464c68240764534437e13c196742598c3db9" exitCode=0 Jul 23 18:00:14.468424 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:14.468319 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5m2x2" event={"ID":"1b816e2d-d250-4c9f-abb2-ff37218ff860","Type":"ContainerDied","Data":"dc5e9cebf5a7e3df0282aef20817464c68240764534437e13c196742598c3db9"} Jul 23 18:00:15.077687 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:15.077410 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:15.137079 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:15.137039 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:15.475794 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:15.475755 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b816e2d-d250-4c9f-abb2-ff37218ff860" containerID="f9fd73ed67eb65f1c2cd7a9dbbe79d7a9b862ce01923b4bce7a911099c96f1e3" exitCode=0 Jul 23 18:00:15.476230 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:15.475806 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5m2x2" event={"ID":"1b816e2d-d250-4c9f-abb2-ff37218ff860","Type":"ContainerDied","Data":"f9fd73ed67eb65f1c2cd7a9dbbe79d7a9b862ce01923b4bce7a911099c96f1e3"} Jul 23 18:00:15.603502 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:15.603466 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:15.603689 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:15.603571 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:00:15.603689 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:15.603646 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls podName:84ca6f4e-056f-4d1f-93a4-2e910ef06f08 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:19.603624413 +0000 UTC m=+41.091116384 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls") pod "dns-default-hzc9d" (UID: "84ca6f4e-056f-4d1f-93a4-2e910ef06f08") : secret "dns-default-metrics-tls" not found Jul 23 18:00:16.569318 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.569244 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2snwn"] Jul 23 18:00:16.588179 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.588154 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh"] Jul 23 18:00:16.588319 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.588290 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.593183 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.593156 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 18:00:16.593340 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.593154 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 18:00:16.593693 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.593677 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 18:00:16.593740 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.593678 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 18:00:16.593956 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.593942 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mq246\"" Jul 23 18:00:16.608584 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.608562 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2snwn"] Jul 23 18:00:16.608692 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.608605 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh"] Jul 23 18:00:16.608692 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.608681 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" Jul 23 18:00:16.611064 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.611046 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 23 18:00:16.611189 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.611079 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 23 18:00:16.611189 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.611093 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-x7j8x\"" Jul 23 18:00:16.611189 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.611082 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 23 18:00:16.611386 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.611378 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 23 18:00:16.617277 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.617256 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:16.711452 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.711426 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-crio-socket\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.711562 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.711467 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.711562 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.711497 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-data-volume\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.711562 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.711521 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.711562 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.711558 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjppn\" (UniqueName: \"kubernetes.io/projected/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-api-access-sjppn\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.711750 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.711630 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k6wp\" (UniqueName: \"kubernetes.io/projected/8152bbb3-da20-460a-b6e7-2a5aa61bee78-kube-api-access-8k6wp\") pod \"kube-storage-version-migrator-operator-7658d9b749-wgqrh\" (UID: \"8152bbb3-da20-460a-b6e7-2a5aa61bee78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" Jul 23 18:00:16.711815 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.711789 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8152bbb3-da20-460a-b6e7-2a5aa61bee78-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-wgqrh\" (UID: \"8152bbb3-da20-460a-b6e7-2a5aa61bee78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" Jul 23 18:00:16.711851 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.711833 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8152bbb3-da20-460a-b6e7-2a5aa61bee78-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-wgqrh\" (UID: \"8152bbb3-da20-460a-b6e7-2a5aa61bee78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" Jul 23 18:00:16.812194 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.812167 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8152bbb3-da20-460a-b6e7-2a5aa61bee78-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-wgqrh\" (UID: \"8152bbb3-da20-460a-b6e7-2a5aa61bee78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" Jul 23 18:00:16.812326 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.812200 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8152bbb3-da20-460a-b6e7-2a5aa61bee78-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-wgqrh\" (UID: \"8152bbb3-da20-460a-b6e7-2a5aa61bee78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" Jul 23 18:00:16.812326 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.812265 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-crio-socket\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.812326 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.812288 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.812326 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.812306 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-data-volume\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.812526 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.812328 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.812526 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:16.812411 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 18:00:16.812526 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:16.812434 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:00:16.812526 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.812434 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjppn\" (UniqueName: \"kubernetes.io/projected/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-api-access-sjppn\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.812526 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:16.812479 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm podName:ccdc7fc1-9212-4c7f-ad21-3916e5496d12 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:17.312460296 +0000 UTC m=+38.799952267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2snwn" (UID: "ccdc7fc1-9212-4c7f-ad21-3916e5496d12") : configmap "kube-rbac-proxy" not found Jul 23 18:00:16.812526 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.812497 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-crio-socket\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.812768 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:16.812654 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls podName:ccdc7fc1-9212-4c7f-ad21-3916e5496d12 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:17.31263292 +0000 UTC m=+38.800124890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2snwn" (UID: "ccdc7fc1-9212-4c7f-ad21-3916e5496d12") : secret "insights-runtime-extractor-tls" not found Jul 23 18:00:16.812768 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.812692 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k6wp\" (UniqueName: \"kubernetes.io/projected/8152bbb3-da20-460a-b6e7-2a5aa61bee78-kube-api-access-8k6wp\") pod \"kube-storage-version-migrator-operator-7658d9b749-wgqrh\" (UID: \"8152bbb3-da20-460a-b6e7-2a5aa61bee78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" Jul 23 18:00:16.812768 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.812734 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-data-volume\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.813124 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.813102 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8152bbb3-da20-460a-b6e7-2a5aa61bee78-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-wgqrh\" (UID: \"8152bbb3-da20-460a-b6e7-2a5aa61bee78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" Jul 23 18:00:16.816131 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.816103 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8152bbb3-da20-460a-b6e7-2a5aa61bee78-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-wgqrh\" (UID: \"8152bbb3-da20-460a-b6e7-2a5aa61bee78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" Jul 23 18:00:16.821897 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.821828 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjppn\" (UniqueName: \"kubernetes.io/projected/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-api-access-sjppn\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:16.822118 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.822095 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k6wp\" (UniqueName: \"kubernetes.io/projected/8152bbb3-da20-460a-b6e7-2a5aa61bee78-kube-api-access-8k6wp\") pod \"kube-storage-version-migrator-operator-7658d9b749-wgqrh\" (UID: \"8152bbb3-da20-460a-b6e7-2a5aa61bee78\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" Jul 23 18:00:16.916335 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:16.916289 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" Jul 23 18:00:17.055943 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.055762 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh"] Jul 23 18:00:17.060962 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:17.060937 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8152bbb3_da20_460a_b6e7_2a5aa61bee78.slice/crio-8e9342a459504a3b25b09f22554c3382099757745c14c24f8c1bdfa95d560a8d WatchSource:0}: Error finding container 8e9342a459504a3b25b09f22554c3382099757745c14c24f8c1bdfa95d560a8d: Status 404 returned error can't find the container with id 8e9342a459504a3b25b09f22554c3382099757745c14c24f8c1bdfa95d560a8d Jul 23 18:00:17.228668 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.228636 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:17.317166 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.317128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:17.317317 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.317173 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:17.317317 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:17.317301 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:00:17.317317 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:17.317311 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 18:00:17.317403 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:17.317369 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls podName:ccdc7fc1-9212-4c7f-ad21-3916e5496d12 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:18.317352161 +0000 UTC m=+39.804844137 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2snwn" (UID: "ccdc7fc1-9212-4c7f-ad21-3916e5496d12") : secret "insights-runtime-extractor-tls" not found Jul 23 18:00:17.317403 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:17.317386 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm podName:ccdc7fc1-9212-4c7f-ad21-3916e5496d12 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:18.317379051 +0000 UTC m=+39.804871020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2snwn" (UID: "ccdc7fc1-9212-4c7f-ad21-3916e5496d12") : configmap "kube-rbac-proxy" not found Jul 23 18:00:17.375788 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.375755 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:17.481459 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.481387 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/0.log" Jul 23 18:00:17.481459 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.481425 2562 generic.go:356] "Generic (PLEG): container finished" podID="ff788e6b-0522-4742-8f8f-d100924612a0" containerID="1a3ea3b29cf0e3beb84463a5045747d362c934117ea08edd35cce21be12fe7a0" exitCode=255 Jul 23 18:00:17.481682 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.481506 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" event={"ID":"ff788e6b-0522-4742-8f8f-d100924612a0","Type":"ContainerDied","Data":"1a3ea3b29cf0e3beb84463a5045747d362c934117ea08edd35cce21be12fe7a0"} Jul 23 18:00:17.481783 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.481767 2562 scope.go:117] "RemoveContainer" containerID="1a3ea3b29cf0e3beb84463a5045747d362c934117ea08edd35cce21be12fe7a0" Jul 23 18:00:17.485145 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.485110 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fsxnc" event={"ID":"0833f914-e35c-4a6d-8a60-0e99269bee04","Type":"ContainerStarted","Data":"0f9f6fd63fb2687560560bc1c8857eb8c95a583faa19f280dd4d90307e99a03c"} Jul 23 18:00:17.488403 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.488377 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5m2x2" event={"ID":"1b816e2d-d250-4c9f-abb2-ff37218ff860","Type":"ContainerStarted","Data":"357b42d6e8444c5a3b5efde7f8a235e8de1e0b191aa560f21981860c99073e97"} Jul 23 18:00:17.489448 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.489430 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" event={"ID":"8152bbb3-da20-460a-b6e7-2a5aa61bee78","Type":"ContainerStarted","Data":"8e9342a459504a3b25b09f22554c3382099757745c14c24f8c1bdfa95d560a8d"} Jul 23 18:00:17.565639 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.565587 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5m2x2" podStartSLOduration=9.293245102 podStartE2EDuration="38.565568989s" totalImagesPullingTime="29.272323887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:17.562737853 +0000 UTC m=+39.050229860" watchObservedRunningTime="2026-07-23 18:00:17.565568989 +0000 UTC m=+39.053060958" Jul 23 18:00:17.600377 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.600329 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fsxnc" podStartSLOduration=3.559257628 podStartE2EDuration="6.600310454s" totalImagesPullingTime="3.041052826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:17.599100244 +0000 UTC m=+39.086592234" watchObservedRunningTime="2026-07-23 18:00:17.600310454 +0000 UTC m=+39.087802445" Jul 23 18:00:17.713853 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.713824 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:17.814149 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:17.814066 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:18.324609 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.324529 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:18.324609 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.324602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:18.324837 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:18.324713 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:00:18.324837 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:18.324749 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 18:00:18.324837 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:18.324796 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls podName:ccdc7fc1-9212-4c7f-ad21-3916e5496d12 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:20.324775109 +0000 UTC m=+41.812267091 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2snwn" (UID: "ccdc7fc1-9212-4c7f-ad21-3916e5496d12") : secret "insights-runtime-extractor-tls" not found Jul 23 18:00:18.324837 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:18.324820 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm podName:ccdc7fc1-9212-4c7f-ad21-3916e5496d12 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:20.324808376 +0000 UTC m=+41.812300349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2snwn" (UID: "ccdc7fc1-9212-4c7f-ad21-3916e5496d12") : configmap "kube-rbac-proxy" not found Jul 23 18:00:18.378913 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.378871 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:18.492934 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.492908 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:00:18.493348 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.493333 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/0.log" Jul 23 18:00:18.493405 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.493368 2562 generic.go:356] "Generic (PLEG): container finished" podID="ff788e6b-0522-4742-8f8f-d100924612a0" containerID="9613699719f70f54b432fc36108432ae10a634e06ae2b56105c2f2860fa7cdd6" exitCode=255 Jul 23 18:00:18.493438 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.493400 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" event={"ID":"ff788e6b-0522-4742-8f8f-d100924612a0","Type":"ContainerDied","Data":"9613699719f70f54b432fc36108432ae10a634e06ae2b56105c2f2860fa7cdd6"} Jul 23 18:00:18.493498 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.493484 2562 scope.go:117] "RemoveContainer" containerID="1a3ea3b29cf0e3beb84463a5045747d362c934117ea08edd35cce21be12fe7a0" Jul 23 18:00:18.493736 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.493717 2562 scope.go:117] "RemoveContainer" containerID="9613699719f70f54b432fc36108432ae10a634e06ae2b56105c2f2860fa7cdd6" Jul 23 18:00:18.493898 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:18.493872 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-9qgsr_openshift-console-operator(ff788e6b-0522-4742-8f8f-d100924612a0)\"" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" podUID="ff788e6b-0522-4742-8f8f-d100924612a0" Jul 23 18:00:18.554040 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.554014 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j"] Jul 23 18:00:18.585308 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.585235 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j"] Jul 23 18:00:18.585435 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.585324 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" Jul 23 18:00:18.588230 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.588197 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 23 18:00:18.588382 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.588203 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 23 18:00:18.588382 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.588276 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 23 18:00:18.588487 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.588457 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-9r796\"" Jul 23 18:00:18.588550 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.588510 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 23 18:00:18.715558 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.715529 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngf5j_b673b420-6fcb-4021-8ea7-1dce6101f1ad/dns-node-resolver/0.log" Jul 23 18:00:18.727774 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.727748 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e422aa7b-a5f0-44d6-8819-d6619b6b455b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-62b6j\" (UID: \"e422aa7b-a5f0-44d6-8819-d6619b6b455b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" Jul 23 18:00:18.727917 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.727850 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e422aa7b-a5f0-44d6-8819-d6619b6b455b-config\") pod \"service-ca-operator-5bcbc94c78-62b6j\" (UID: \"e422aa7b-a5f0-44d6-8819-d6619b6b455b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" Jul 23 18:00:18.727917 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.727902 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhlsq\" (UniqueName: \"kubernetes.io/projected/e422aa7b-a5f0-44d6-8819-d6619b6b455b-kube-api-access-fhlsq\") pod \"service-ca-operator-5bcbc94c78-62b6j\" (UID: \"e422aa7b-a5f0-44d6-8819-d6619b6b455b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" Jul 23 18:00:18.829092 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.829056 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e422aa7b-a5f0-44d6-8819-d6619b6b455b-config\") pod \"service-ca-operator-5bcbc94c78-62b6j\" (UID: \"e422aa7b-a5f0-44d6-8819-d6619b6b455b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" Jul 23 18:00:18.829263 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.829104 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhlsq\" (UniqueName: \"kubernetes.io/projected/e422aa7b-a5f0-44d6-8819-d6619b6b455b-kube-api-access-fhlsq\") pod \"service-ca-operator-5bcbc94c78-62b6j\" (UID: \"e422aa7b-a5f0-44d6-8819-d6619b6b455b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" Jul 23 18:00:18.829263 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.829175 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e422aa7b-a5f0-44d6-8819-d6619b6b455b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-62b6j\" (UID: \"e422aa7b-a5f0-44d6-8819-d6619b6b455b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" Jul 23 18:00:18.829664 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.829641 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e422aa7b-a5f0-44d6-8819-d6619b6b455b-config\") pod \"service-ca-operator-5bcbc94c78-62b6j\" (UID: \"e422aa7b-a5f0-44d6-8819-d6619b6b455b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" Jul 23 18:00:18.832270 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.832253 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e422aa7b-a5f0-44d6-8819-d6619b6b455b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-62b6j\" (UID: \"e422aa7b-a5f0-44d6-8819-d6619b6b455b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" Jul 23 18:00:18.837253 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.837175 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhlsq\" (UniqueName: \"kubernetes.io/projected/e422aa7b-a5f0-44d6-8819-d6619b6b455b-kube-api-access-fhlsq\") pod \"service-ca-operator-5bcbc94c78-62b6j\" (UID: \"e422aa7b-a5f0-44d6-8819-d6619b6b455b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" Jul 23 18:00:18.894038 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:18.893999 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" Jul 23 18:00:19.156598 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.156505 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-gzxx6"] Jul 23 18:00:19.178788 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.178762 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-gzxx6"] Jul 23 18:00:19.178923 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.178865 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-gzxx6" Jul 23 18:00:19.181759 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.181731 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-8wwds\"" Jul 23 18:00:19.316161 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.316132 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ljb4k_b939e580-7ce8-4e0b-a2c6-6e55f65e25d8/node-ca/0.log" Jul 23 18:00:19.333623 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.333590 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btj6b\" (UniqueName: \"kubernetes.io/projected/239752c6-7354-411d-945c-2fcdf6063ae0-kube-api-access-btj6b\") pod \"network-check-source-7bc5b6bf85-gzxx6\" (UID: \"239752c6-7354-411d-945c-2fcdf6063ae0\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-gzxx6" Jul 23 18:00:19.434743 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.434707 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btj6b\" (UniqueName: \"kubernetes.io/projected/239752c6-7354-411d-945c-2fcdf6063ae0-kube-api-access-btj6b\") pod \"network-check-source-7bc5b6bf85-gzxx6\" (UID: \"239752c6-7354-411d-945c-2fcdf6063ae0\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-gzxx6" Jul 23 18:00:19.444521 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.444494 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btj6b\" (UniqueName: \"kubernetes.io/projected/239752c6-7354-411d-945c-2fcdf6063ae0-kube-api-access-btj6b\") pod \"network-check-source-7bc5b6bf85-gzxx6\" (UID: \"239752c6-7354-411d-945c-2fcdf6063ae0\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-gzxx6" Jul 23 18:00:19.459401 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.459378 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j"] Jul 23 18:00:19.462358 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:19.462325 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode422aa7b_a5f0_44d6_8819_d6619b6b455b.slice/crio-780478f9e98c9b9d38f811e2d075ed08813661c38be779943cd0680be2115d19 WatchSource:0}: Error finding container 780478f9e98c9b9d38f811e2d075ed08813661c38be779943cd0680be2115d19: Status 404 returned error can't find the container with id 780478f9e98c9b9d38f811e2d075ed08813661c38be779943cd0680be2115d19 Jul 23 18:00:19.490082 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.490053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-gzxx6" Jul 23 18:00:19.497430 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.497408 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:00:19.497755 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.497740 2562 scope.go:117] "RemoveContainer" containerID="9613699719f70f54b432fc36108432ae10a634e06ae2b56105c2f2860fa7cdd6" Jul 23 18:00:19.497910 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:19.497891 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-9qgsr_openshift-console-operator(ff788e6b-0522-4742-8f8f-d100924612a0)\"" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" podUID="ff788e6b-0522-4742-8f8f-d100924612a0" Jul 23 18:00:19.498542 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.498516 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" event={"ID":"e422aa7b-a5f0-44d6-8819-d6619b6b455b","Type":"ContainerStarted","Data":"780478f9e98c9b9d38f811e2d075ed08813661c38be779943cd0680be2115d19"} Jul 23 18:00:19.595028 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.594999 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65f9d79445-h5c4b"] Jul 23 18:00:19.609171 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.609148 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.611904 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.611881 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 18:00:19.613857 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.613796 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 18:00:19.614012 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.613981 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4ddxk\"" Jul 23 18:00:19.614132 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.614102 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 18:00:19.614937 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.614913 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65f9d79445-h5c4b"] Jul 23 18:00:19.619835 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.619817 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 18:00:19.628781 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.628759 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-gzxx6"] Jul 23 18:00:19.636907 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.636884 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:19.637154 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:19.637134 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:00:19.637263 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:19.637196 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls podName:84ca6f4e-056f-4d1f-93a4-2e910ef06f08 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:27.637179043 +0000 UTC m=+49.124671015 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls") pod "dns-default-hzc9d" (UID: "84ca6f4e-056f-4d1f-93a4-2e910ef06f08") : secret "dns-default-metrics-tls" not found Jul 23 18:00:19.735346 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:19.735311 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239752c6_7354_411d_945c_2fcdf6063ae0.slice/crio-7096403c43f515b261d44871b1b8edcca4fc3d868c7c9cb5eb1a6d2a06b2311e WatchSource:0}: Error finding container 7096403c43f515b261d44871b1b8edcca4fc3d868c7c9cb5eb1a6d2a06b2311e: Status 404 returned error can't find the container with id 7096403c43f515b261d44871b1b8edcca4fc3d868c7c9cb5eb1a6d2a06b2311e Jul 23 18:00:19.737997 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.737977 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-ca-trust-extracted\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.738069 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.738018 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-installation-pull-secrets\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.738069 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.738047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-certificates\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.738171 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.738125 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22dst\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-kube-api-access-22dst\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.738245 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.738187 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-trusted-ca\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.738307 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.738253 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-image-registry-private-configuration\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.738307 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.738284 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-bound-sa-token\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.738402 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.738316 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.842588 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.842542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-ca-trust-extracted\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.842706 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.842647 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-installation-pull-secrets\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.842760 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.842717 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-certificates\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.842876 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.842856 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-ca-trust-extracted\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.842931 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.842872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22dst\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-kube-api-access-22dst\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.842988 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.842937 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-trusted-ca\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.842988 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.842968 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-image-registry-private-configuration\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.843082 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.842993 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-bound-sa-token\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.843082 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.843020 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.843187 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:19.843163 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 18:00:19.843187 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:19.843183 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f9d79445-h5c4b: secret "image-registry-tls" not found Jul 23 18:00:19.843289 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:19.843261 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls podName:0806e87e-c116-4fa0-b9ae-4f657a08b6b1 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:20.343242154 +0000 UTC m=+41.830734130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls") pod "image-registry-65f9d79445-h5c4b" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1") : secret "image-registry-tls" not found Jul 23 18:00:19.843347 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.843284 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-certificates\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.843814 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.843793 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-trusted-ca\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.845044 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.845011 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-installation-pull-secrets\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.845243 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.845206 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-image-registry-private-configuration\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.853403 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.853379 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22dst\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-kube-api-access-22dst\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.853578 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.853562 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-bound-sa-token\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:19.916068 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:19.916034 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fsxnc_0833f914-e35c-4a6d-8a60-0e99269bee04/serve-healthcheck-canary/0.log" Jul 23 18:00:20.346580 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:20.346550 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:20.346774 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:20.346671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:20.346774 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:20.346724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:20.346774 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:20.346740 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 18:00:20.346774 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:20.346765 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f9d79445-h5c4b: secret "image-registry-tls" not found Jul 23 18:00:20.346955 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:20.346801 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 18:00:20.346955 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:20.346814 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:00:20.346955 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:20.346832 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls podName:0806e87e-c116-4fa0-b9ae-4f657a08b6b1 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:21.346811556 +0000 UTC m=+42.834303529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls") pod "image-registry-65f9d79445-h5c4b" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1") : secret "image-registry-tls" not found Jul 23 18:00:20.346955 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:20.346855 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls podName:ccdc7fc1-9212-4c7f-ad21-3916e5496d12 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:24.346839885 +0000 UTC m=+45.834331858 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2snwn" (UID: "ccdc7fc1-9212-4c7f-ad21-3916e5496d12") : secret "insights-runtime-extractor-tls" not found Jul 23 18:00:20.346955 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:20.346873 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm podName:ccdc7fc1-9212-4c7f-ad21-3916e5496d12 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:24.34686483 +0000 UTC m=+45.834356799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2snwn" (UID: "ccdc7fc1-9212-4c7f-ad21-3916e5496d12") : configmap "kube-rbac-proxy" not found Jul 23 18:00:20.503829 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:20.503341 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" event={"ID":"8152bbb3-da20-460a-b6e7-2a5aa61bee78","Type":"ContainerStarted","Data":"f90a543860ff84f72d3b84210675f5df2324ab8291e871613daaffede09fbf9e"} Jul 23 18:00:20.504666 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:20.504637 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-gzxx6" event={"ID":"239752c6-7354-411d-945c-2fcdf6063ae0","Type":"ContainerStarted","Data":"7096403c43f515b261d44871b1b8edcca4fc3d868c7c9cb5eb1a6d2a06b2311e"} Jul 23 18:00:20.521855 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:20.521806 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" podStartSLOduration=1.813504545 podStartE2EDuration="4.521789883s" totalImagesPullingTime="2.708285338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:20.52142434 +0000 UTC m=+42.008916343" watchObservedRunningTime="2026-07-23 18:00:20.521789883 +0000 UTC m=+42.009281873" Jul 23 18:00:21.357459 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:21.357407 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:21.357926 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:21.357638 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 18:00:21.357926 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:21.357656 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f9d79445-h5c4b: secret "image-registry-tls" not found Jul 23 18:00:21.357926 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:21.357753 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls podName:0806e87e-c116-4fa0-b9ae-4f657a08b6b1 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:23.357731349 +0000 UTC m=+44.845223321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls") pod "image-registry-65f9d79445-h5c4b" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1") : secret "image-registry-tls" not found Jul 23 18:00:22.512595 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:22.512552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" event={"ID":"e422aa7b-a5f0-44d6-8819-d6619b6b455b","Type":"ContainerStarted","Data":"74e665d7cf6750e1b8d6a6b3a164edc7ef89c23ee815fa8b1f5cf60017f5bb81"} Jul 23 18:00:22.532372 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:22.532297 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" podStartSLOduration=2.011488976 podStartE2EDuration="4.53227681s" totalImagesPullingTime="2.520787834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:00:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:22.531630488 +0000 UTC m=+44.019122482" watchObservedRunningTime="2026-07-23 18:00:22.53227681 +0000 UTC m=+44.019768801" Jul 23 18:00:22.809743 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:22.809600 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:22.879894 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:22.879858 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:22.880083 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:22.879924 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:22.880357 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:22.880333 2562 scope.go:117] "RemoveContainer" containerID="9613699719f70f54b432fc36108432ae10a634e06ae2b56105c2f2860fa7cdd6" Jul 23 18:00:22.880527 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:22.880501 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-9qgsr_openshift-console-operator(ff788e6b-0522-4742-8f8f-d100924612a0)\"" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" podUID="ff788e6b-0522-4742-8f8f-d100924612a0" Jul 23 18:00:22.888879 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:22.888854 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:23.210556 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:23.210512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:23.322385 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:23.322332 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:23.374908 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:23.374872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:23.375057 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:23.374999 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 18:00:23.375057 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:23.375012 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f9d79445-h5c4b: secret "image-registry-tls" not found Jul 23 18:00:23.375132 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:23.375068 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls podName:0806e87e-c116-4fa0-b9ae-4f657a08b6b1 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:27.375049162 +0000 UTC m=+48.862541134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls") pod "image-registry-65f9d79445-h5c4b" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1") : secret "image-registry-tls" not found Jul 23 18:00:24.385486 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:24.385441 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:24.385866 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:24.385512 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:24.385866 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:24.385598 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 18:00:24.385866 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:24.385598 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:00:24.385866 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:24.385659 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm podName:ccdc7fc1-9212-4c7f-ad21-3916e5496d12 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:32.385641529 +0000 UTC m=+53.873133504 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2snwn" (UID: "ccdc7fc1-9212-4c7f-ad21-3916e5496d12") : configmap "kube-rbac-proxy" not found Jul 23 18:00:24.385866 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:24.385688 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls podName:ccdc7fc1-9212-4c7f-ad21-3916e5496d12 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:32.385676058 +0000 UTC m=+53.873168028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2snwn" (UID: "ccdc7fc1-9212-4c7f-ad21-3916e5496d12") : secret "insights-runtime-extractor-tls" not found Jul 23 18:00:24.505374 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:24.505338 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:24.519914 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:24.519877 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-gzxx6" event={"ID":"239752c6-7354-411d-945c-2fcdf6063ae0","Type":"ContainerStarted","Data":"43d9774b3ee7960fbec87bb60513d882e1f9e3df32bc8028a46b2a6e1a6ec53e"} Jul 23 18:00:24.543005 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:24.539420 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-gzxx6" podStartSLOduration=1.567490652 podStartE2EDuration="5.539394686s" totalImagesPullingTime="3.971904034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:24.537616546 +0000 UTC m=+46.025108538" watchObservedRunningTime="2026-07-23 18:00:24.539394686 +0000 UTC m=+46.026886678" Jul 23 18:00:24.949842 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:24.949809 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:25.049065 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:25.049031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:25.328284 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:25.328183 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:26.606585 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:26.606554 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:27.412755 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:27.412720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:27.412943 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:27.412874 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 18:00:27.412943 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:27.412895 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f9d79445-h5c4b: secret "image-registry-tls" not found Jul 23 18:00:27.413018 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:27.412957 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls podName:0806e87e-c116-4fa0-b9ae-4f657a08b6b1 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:35.412940605 +0000 UTC m=+56.900432574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls") pod "image-registry-65f9d79445-h5c4b" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1") : secret "image-registry-tls" not found Jul 23 18:00:27.672051 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:27.671944 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:27.715184 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:27.715153 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:27.715351 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:27.715322 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:00:27.715402 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:27.715391 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls podName:84ca6f4e-056f-4d1f-93a4-2e910ef06f08 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:43.715375123 +0000 UTC m=+65.202867095 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls") pod "dns-default-hzc9d" (UID: "84ca6f4e-056f-4d1f-93a4-2e910ef06f08") : secret "dns-default-metrics-tls" not found Jul 23 18:00:29.786915 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:29.786851 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:30.147466 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:30.147361 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:30.441267 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:30.441169 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:30.600466 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:30.600408 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:31.181366 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:31.181334 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:31.440184 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:31.440100 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:31.540393 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:31.540362 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:31.811910 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:31.811879 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:31.938935 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:31.938904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:32.078626 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:32.078546 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:32.452356 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:32.452320 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:32.452725 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:32.452376 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:32.452725 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:32.452470 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:00:32.452725 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:00:32.452539 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls podName:ccdc7fc1-9212-4c7f-ad21-3916e5496d12 nodeName:}" failed. No retries permitted until 2026-07-23 18:00:48.452522271 +0000 UTC m=+69.940014242 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2snwn" (UID: "ccdc7fc1-9212-4c7f-ad21-3916e5496d12") : secret "insights-runtime-extractor-tls" not found Jul 23 18:00:32.452913 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:32.452892 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:32.481081 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:32.481049 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:33.421619 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:33.421586 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:33.681419 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:33.681335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:33.895889 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:33.895857 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:34.085039 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:34.085005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:34.244246 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:34.244195 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:34.318279 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:34.318242 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:34.824186 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:34.824157 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:34.925476 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:34.925441 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:35.475466 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:35.475420 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:35.477938 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:35.477918 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls\") pod \"image-registry-65f9d79445-h5c4b\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:35.481442 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:35.481415 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:35.520417 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:35.520382 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:35.654709 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:35.654676 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65f9d79445-h5c4b"] Jul 23 18:00:35.657125 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:35.657100 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0806e87e_c116_4fa0_b9ae_4f657a08b6b1.slice/crio-453436301db69e7f01062e15d3d30454f7b595a91ef8782aeab838648c5fccd4 WatchSource:0}: Error finding container 453436301db69e7f01062e15d3d30454f7b595a91ef8782aeab838648c5fccd4: Status 404 returned error can't find the container with id 453436301db69e7f01062e15d3d30454f7b595a91ef8782aeab838648c5fccd4 Jul 23 18:00:35.693151 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:35.693083 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:35.797468 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:35.797376 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:35.902865 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:35.902834 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:36.070203 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:36.070124 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:36.271104 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:36.271072 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:36.552345 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:36.552309 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" event={"ID":"0806e87e-c116-4fa0-b9ae-4f657a08b6b1","Type":"ContainerStarted","Data":"5c7136a76500b01de6e52e13061428671af361cf5c428930c5296afcbccdfe17"} Jul 23 18:00:36.552528 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:36.552387 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" event={"ID":"0806e87e-c116-4fa0-b9ae-4f657a08b6b1","Type":"ContainerStarted","Data":"453436301db69e7f01062e15d3d30454f7b595a91ef8782aeab838648c5fccd4"} Jul 23 18:00:36.552528 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:36.552432 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:00:36.576716 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:36.576665 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" podStartSLOduration=17.576651383 podStartE2EDuration="17.576651383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:36.575899459 +0000 UTC m=+58.063391452" watchObservedRunningTime="2026-07-23 18:00:36.576651383 +0000 UTC m=+58.064143374" Jul 23 18:00:37.001954 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:37.001917 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:37.581996 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:37.581962 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:37.701300 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:37.701263 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:37.785278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:37.785244 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:38.072577 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:38.072533 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:38.124865 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:38.124833 2562 scope.go:117] "RemoveContainer" containerID="9613699719f70f54b432fc36108432ae10a634e06ae2b56105c2f2860fa7cdd6" Jul 23 18:00:38.456914 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:38.456889 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7stv" Jul 23 18:00:38.469419 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:38.469381 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:38.560561 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:38.560532 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:00:38.560727 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:38.560606 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" event={"ID":"ff788e6b-0522-4742-8f8f-d100924612a0","Type":"ContainerStarted","Data":"f085dd9562c9d4135899930a4bc6352ab8947d9207a9362633644ef52e92f720"} Jul 23 18:00:38.560903 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:38.560885 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:38.585460 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:38.585405 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" podStartSLOduration=23.540539768 podStartE2EDuration="26.585387748s" totalImagesPullingTime="3.04484798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:00:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:38.581316429 +0000 UTC m=+60.068808421" watchObservedRunningTime="2026-07-23 18:00:38.585387748 +0000 UTC m=+60.072879740" Jul 23 18:00:38.903000 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:38.902928 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-9qgsr" Jul 23 18:00:39.073141 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:39.073107 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:39.175781 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:39.175693 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:39.284529 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:39.284480 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:39.506860 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:39.506818 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:39.617098 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:39.617067 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:39.932421 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:39.932329 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:40.054276 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:40.054245 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:40.209089 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:40.209060 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:40.727270 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:40.727239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:40.831768 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:40.831730 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:40.932065 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:40.932016 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:41.035661 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:41.035553 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:41.487441 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:41.487416 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:42.277542 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:42.277513 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:42.659868 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:42.659763 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:42.730489 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:42.730455 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:43.741310 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.741261 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:43.743656 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.743633 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84ca6f4e-056f-4d1f-93a4-2e910ef06f08-metrics-tls\") pod \"dns-default-hzc9d\" (UID: \"84ca6f4e-056f-4d1f-93a4-2e910ef06f08\") " pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:43.842161 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.842110 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 18:00:43.842387 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.842231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:43.845155 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.845130 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 18:00:43.845817 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.845799 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 18:00:43.855032 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.855005 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/822c461b-5e7b-4abb-a090-edacc6b4392c-original-pull-secret\") pod \"global-pull-secret-syncer-h62kj\" (UID: \"822c461b-5e7b-4abb-a090-edacc6b4392c\") " pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 18:00:43.855152 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.855038 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daeb5059-d59d-457e-a5c5-d8545f19b19e-metrics-certs\") pod \"network-metrics-daemon-88l5f\" (UID: \"daeb5059-d59d-457e-a5c5-d8545f19b19e\") " pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:43.940146 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.940117 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hv4kc\"" Jul 23 18:00:43.942526 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.942504 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fhbm\" (UniqueName: \"kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm\") pod \"network-check-target-l8nhz\" (UID: \"4178e19f-ad0f-4366-8f06-471cb052caf5\") " pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:43.945019 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.944998 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fhbm\" (UniqueName: \"kubernetes.io/projected/4178e19f-ad0f-4366-8f06-471cb052caf5-kube-api-access-4fhbm\") pod \"network-check-target-l8nhz\" (UID: \"4178e19f-ad0f-4366-8f06-471cb052caf5\") " pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:43.947627 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.947604 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88l5f" Jul 23 18:00:43.947727 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.947686 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zbh2j\"" Jul 23 18:00:43.949981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.949961 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h62kj" Jul 23 18:00:43.956141 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.956122 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:43.956802 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.956786 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lb24m\"" Jul 23 18:00:43.964806 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:43.964763 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:44.109892 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:44.109853 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h62kj"] Jul 23 18:00:44.113144 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:44.113112 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod822c461b_5e7b_4abb_a090_edacc6b4392c.slice/crio-102b1a079363d6c2515fd9dc431b75d6457f99013c85fc35dbe5bacfb1c16c97 WatchSource:0}: Error finding container 102b1a079363d6c2515fd9dc431b75d6457f99013c85fc35dbe5bacfb1c16c97: Status 404 returned error can't find the container with id 102b1a079363d6c2515fd9dc431b75d6457f99013c85fc35dbe5bacfb1c16c97 Jul 23 18:00:44.335197 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:44.335174 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l8nhz"] Jul 23 18:00:44.337501 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:44.337472 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4178e19f_ad0f_4366_8f06_471cb052caf5.slice/crio-da25d1641b82971e40e597bc34348c9969d4c2b9a25c63e47922f70a1a4e7d7c WatchSource:0}: Error finding container da25d1641b82971e40e597bc34348c9969d4c2b9a25c63e47922f70a1a4e7d7c: Status 404 returned error can't find the container with id da25d1641b82971e40e597bc34348c9969d4c2b9a25c63e47922f70a1a4e7d7c Jul 23 18:00:44.340524 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:44.340496 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-88l5f"] Jul 23 18:00:44.342518 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:44.342499 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hzc9d"] Jul 23 18:00:44.344125 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:44.344102 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaeb5059_d59d_457e_a5c5_d8545f19b19e.slice/crio-a62f97ae972ff2af5a3db173c8f559b7f9667230299478aeb8d6a5e49703a5e5 WatchSource:0}: Error finding container a62f97ae972ff2af5a3db173c8f559b7f9667230299478aeb8d6a5e49703a5e5: Status 404 returned error can't find the container with id a62f97ae972ff2af5a3db173c8f559b7f9667230299478aeb8d6a5e49703a5e5 Jul 23 18:00:44.345179 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:44.345160 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ca6f4e_056f_4d1f_93a4_2e910ef06f08.slice/crio-88f98819630f19a7dacf7c9db332b21738a288d1453f1557703b784297e2d06c WatchSource:0}: Error finding container 88f98819630f19a7dacf7c9db332b21738a288d1453f1557703b784297e2d06c: Status 404 returned error can't find the container with id 88f98819630f19a7dacf7c9db332b21738a288d1453f1557703b784297e2d06c Jul 23 18:00:44.581605 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:44.581568 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l8nhz" event={"ID":"4178e19f-ad0f-4366-8f06-471cb052caf5","Type":"ContainerStarted","Data":"ef75ce0348995584f4998fad13ef542e26295fd893eacb6d16b0cd8cbd5a5536"} Jul 23 18:00:44.581798 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:44.581617 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l8nhz" event={"ID":"4178e19f-ad0f-4366-8f06-471cb052caf5","Type":"ContainerStarted","Data":"da25d1641b82971e40e597bc34348c9969d4c2b9a25c63e47922f70a1a4e7d7c"} Jul 23 18:00:44.581798 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:44.581687 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:00:44.582875 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:44.582849 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h62kj" event={"ID":"822c461b-5e7b-4abb-a090-edacc6b4392c","Type":"ContainerStarted","Data":"102b1a079363d6c2515fd9dc431b75d6457f99013c85fc35dbe5bacfb1c16c97"} Jul 23 18:00:44.583916 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:44.583898 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hzc9d" event={"ID":"84ca6f4e-056f-4d1f-93a4-2e910ef06f08","Type":"ContainerStarted","Data":"88f98819630f19a7dacf7c9db332b21738a288d1453f1557703b784297e2d06c"} Jul 23 18:00:44.588819 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:44.588747 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-88l5f" event={"ID":"daeb5059-d59d-457e-a5c5-d8545f19b19e","Type":"ContainerStarted","Data":"a62f97ae972ff2af5a3db173c8f559b7f9667230299478aeb8d6a5e49703a5e5"} Jul 23 18:00:44.598953 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:44.598899 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l8nhz" podStartSLOduration=65.598885529 podStartE2EDuration="1m5.598885529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:44.597680398 +0000 UTC m=+66.085172391" watchObservedRunningTime="2026-07-23 18:00:44.598885529 +0000 UTC m=+66.086377498" Jul 23 18:00:46.181637 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.181577 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:46.501680 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.501363 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl"] Jul 23 18:00:46.517347 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.517302 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd"] Jul 23 18:00:46.518935 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.518033 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:46.527829 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.527549 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 23 18:00:46.527829 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.527663 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 23 18:00:46.527829 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.527722 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 23 18:00:46.533461 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.533435 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd" Jul 23 18:00:46.539686 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.539418 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ztvxf\"" Jul 23 18:00:46.539686 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.539566 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 23 18:00:46.540054 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.540031 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 23 18:00:46.540336 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.540309 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 23 18:00:46.562445 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.562407 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65f9d79445-h5c4b"] Jul 23 18:00:46.566123 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.566044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/570457cc-c6ec-4e45-8a1b-cb87156bd700-klusterlet-config\") pod \"klusterlet-addon-workmgr-6dcdb6c66d-hnlnl\" (UID: \"570457cc-c6ec-4e45-8a1b-cb87156bd700\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:46.566329 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.566155 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4efa0791-cff4-4eef-9f3a-2a584c874ba5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-jhzbd\" (UID: \"4efa0791-cff4-4eef-9f3a-2a584c874ba5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd" Jul 23 18:00:46.566329 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.566199 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/570457cc-c6ec-4e45-8a1b-cb87156bd700-tmp\") pod \"klusterlet-addon-workmgr-6dcdb6c66d-hnlnl\" (UID: \"570457cc-c6ec-4e45-8a1b-cb87156bd700\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:46.566329 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.566265 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwt5r\" (UniqueName: \"kubernetes.io/projected/570457cc-c6ec-4e45-8a1b-cb87156bd700-kube-api-access-cwt5r\") pod \"klusterlet-addon-workmgr-6dcdb6c66d-hnlnl\" (UID: \"570457cc-c6ec-4e45-8a1b-cb87156bd700\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:46.566329 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.566316 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4efa0791-cff4-4eef-9f3a-2a584c874ba5-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-jhzbd\" (UID: \"4efa0791-cff4-4eef-9f3a-2a584c874ba5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd" Jul 23 18:00:46.599401 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.599239 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-88l5f" event={"ID":"daeb5059-d59d-457e-a5c5-d8545f19b19e","Type":"ContainerStarted","Data":"6f7bc3030fd2d652abcfca0ea532c38729c7ed8a56d4d3acce9852ed34d730e7"} Jul 23 18:00:46.599401 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.599309 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-88l5f" event={"ID":"daeb5059-d59d-457e-a5c5-d8545f19b19e","Type":"ContainerStarted","Data":"68faaa7f46b91b7c8b26fee5dd0baa8128cb11da21f7814c1401b714b2e6ba3f"} Jul 23 18:00:46.613110 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.613070 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl"] Jul 23 18:00:46.614682 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.614645 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65749cb4f-4xvjg"] Jul 23 18:00:46.637660 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.637501 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd"] Jul 23 18:00:46.637660 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.637628 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.651147 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.651117 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65749cb4f-4xvjg"] Jul 23 18:00:46.668264 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.667568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwt5r\" (UniqueName: \"kubernetes.io/projected/570457cc-c6ec-4e45-8a1b-cb87156bd700-kube-api-access-cwt5r\") pod \"klusterlet-addon-workmgr-6dcdb6c66d-hnlnl\" (UID: \"570457cc-c6ec-4e45-8a1b-cb87156bd700\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:46.668264 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.667666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4efa0791-cff4-4eef-9f3a-2a584c874ba5-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-jhzbd\" (UID: \"4efa0791-cff4-4eef-9f3a-2a584c874ba5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd" Jul 23 18:00:46.668264 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.667701 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/570457cc-c6ec-4e45-8a1b-cb87156bd700-klusterlet-config\") pod \"klusterlet-addon-workmgr-6dcdb6c66d-hnlnl\" (UID: \"570457cc-c6ec-4e45-8a1b-cb87156bd700\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:46.668264 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.667796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4efa0791-cff4-4eef-9f3a-2a584c874ba5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-jhzbd\" (UID: \"4efa0791-cff4-4eef-9f3a-2a584c874ba5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd" Jul 23 18:00:46.668264 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.667845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/570457cc-c6ec-4e45-8a1b-cb87156bd700-tmp\") pod \"klusterlet-addon-workmgr-6dcdb6c66d-hnlnl\" (UID: \"570457cc-c6ec-4e45-8a1b-cb87156bd700\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:46.668990 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.668935 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4efa0791-cff4-4eef-9f3a-2a584c874ba5-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-jhzbd\" (UID: \"4efa0791-cff4-4eef-9f3a-2a584c874ba5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd" Jul 23 18:00:46.669369 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.669343 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/570457cc-c6ec-4e45-8a1b-cb87156bd700-tmp\") pod \"klusterlet-addon-workmgr-6dcdb6c66d-hnlnl\" (UID: \"570457cc-c6ec-4e45-8a1b-cb87156bd700\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:46.671093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.671073 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/570457cc-c6ec-4e45-8a1b-cb87156bd700-klusterlet-config\") pod \"klusterlet-addon-workmgr-6dcdb6c66d-hnlnl\" (UID: \"570457cc-c6ec-4e45-8a1b-cb87156bd700\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:46.672720 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.672681 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4efa0791-cff4-4eef-9f3a-2a584c874ba5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-jhzbd\" (UID: \"4efa0791-cff4-4eef-9f3a-2a584c874ba5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd" Jul 23 18:00:46.685873 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.685815 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-88l5f" podStartSLOduration=65.944881279 podStartE2EDuration="1m7.68579403s" totalImagesPullingTime="1.740912751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:46.683872627 +0000 UTC m=+68.171364619" watchObservedRunningTime="2026-07-23 18:00:46.68579403 +0000 UTC m=+68.173286036" Jul 23 18:00:46.688965 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.688938 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwt5r\" (UniqueName: \"kubernetes.io/projected/570457cc-c6ec-4e45-8a1b-cb87156bd700-kube-api-access-cwt5r\") pod \"klusterlet-addon-workmgr-6dcdb6c66d-hnlnl\" (UID: \"570457cc-c6ec-4e45-8a1b-cb87156bd700\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:46.751628 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.751568 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:46.768745 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.768696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02000744-cf65-4f6e-8f67-40b30f0d9cc3-image-registry-private-configuration\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.769051 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.769004 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02000744-cf65-4f6e-8f67-40b30f0d9cc3-ca-trust-extracted\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.770181 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.769091 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02000744-cf65-4f6e-8f67-40b30f0d9cc3-registry-tls\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.770181 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.769122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02000744-cf65-4f6e-8f67-40b30f0d9cc3-registry-certificates\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.770181 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.769149 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02000744-cf65-4f6e-8f67-40b30f0d9cc3-trusted-ca\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.770181 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.769174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdgpt\" (UniqueName: \"kubernetes.io/projected/02000744-cf65-4f6e-8f67-40b30f0d9cc3-kube-api-access-sdgpt\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.770181 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.769263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02000744-cf65-4f6e-8f67-40b30f0d9cc3-installation-pull-secrets\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.770181 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.769304 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02000744-cf65-4f6e-8f67-40b30f0d9cc3-bound-sa-token\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.830924 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.830680 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:46.830924 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.830805 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:46.846818 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.846786 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd" Jul 23 18:00:46.870593 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.870551 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02000744-cf65-4f6e-8f67-40b30f0d9cc3-image-registry-private-configuration\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.870784 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.870599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02000744-cf65-4f6e-8f67-40b30f0d9cc3-ca-trust-extracted\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.870784 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.870668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02000744-cf65-4f6e-8f67-40b30f0d9cc3-registry-tls\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.870784 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.870710 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02000744-cf65-4f6e-8f67-40b30f0d9cc3-registry-certificates\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.870784 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.870737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02000744-cf65-4f6e-8f67-40b30f0d9cc3-trusted-ca\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.870784 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.870760 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdgpt\" (UniqueName: \"kubernetes.io/projected/02000744-cf65-4f6e-8f67-40b30f0d9cc3-kube-api-access-sdgpt\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.871041 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.870809 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02000744-cf65-4f6e-8f67-40b30f0d9cc3-installation-pull-secrets\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.871041 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.870838 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02000744-cf65-4f6e-8f67-40b30f0d9cc3-bound-sa-token\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.871142 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.871083 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02000744-cf65-4f6e-8f67-40b30f0d9cc3-ca-trust-extracted\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.872107 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.872068 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02000744-cf65-4f6e-8f67-40b30f0d9cc3-trusted-ca\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.872107 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.872079 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02000744-cf65-4f6e-8f67-40b30f0d9cc3-registry-certificates\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.874332 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.874275 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02000744-cf65-4f6e-8f67-40b30f0d9cc3-image-registry-private-configuration\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.874332 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.874296 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02000744-cf65-4f6e-8f67-40b30f0d9cc3-installation-pull-secrets\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.874687 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.874586 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02000744-cf65-4f6e-8f67-40b30f0d9cc3-registry-tls\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.881884 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.881816 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02000744-cf65-4f6e-8f67-40b30f0d9cc3-bound-sa-token\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.887742 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.887651 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdgpt\" (UniqueName: \"kubernetes.io/projected/02000744-cf65-4f6e-8f67-40b30f0d9cc3-kube-api-access-sdgpt\") pod \"image-registry-65749cb4f-4xvjg\" (UID: \"02000744-cf65-4f6e-8f67-40b30f0d9cc3\") " pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.952335 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.952301 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:46.988305 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:46.988270 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:47.022240 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:47.021383 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl"] Jul 23 18:00:47.075608 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:47.075537 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd"] Jul 23 18:00:47.117319 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:47.117189 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:47.195359 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:47.195325 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65749cb4f-4xvjg"] Jul 23 18:00:47.575350 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:47.575314 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4efa0791_cff4_4eef_9f3a_2a584c874ba5.slice/crio-82d48f8899bdb1e838dd7f9b6ecc92e02ee430a121b8b6892d41ba843d9bc463 WatchSource:0}: Error finding container 82d48f8899bdb1e838dd7f9b6ecc92e02ee430a121b8b6892d41ba843d9bc463: Status 404 returned error can't find the container with id 82d48f8899bdb1e838dd7f9b6ecc92e02ee430a121b8b6892d41ba843d9bc463 Jul 23 18:00:47.575761 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:47.575739 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570457cc_c6ec_4e45_8a1b_cb87156bd700.slice/crio-e7ecf24906c65d052fb1915f5946ec6e64c272a0cbf9ba2f14072b72e6286ff7 WatchSource:0}: Error finding container e7ecf24906c65d052fb1915f5946ec6e64c272a0cbf9ba2f14072b72e6286ff7: Status 404 returned error can't find the container with id e7ecf24906c65d052fb1915f5946ec6e64c272a0cbf9ba2f14072b72e6286ff7 Jul 23 18:00:47.576655 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:47.576562 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02000744_cf65_4f6e_8f67_40b30f0d9cc3.slice/crio-aae2cbd2625fde8187ae0d5947702407d02347a490006d69087644c66ebd22f1 WatchSource:0}: Error finding container aae2cbd2625fde8187ae0d5947702407d02347a490006d69087644c66ebd22f1: Status 404 returned error can't find the container with id aae2cbd2625fde8187ae0d5947702407d02347a490006d69087644c66ebd22f1 Jul 23 18:00:47.605172 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:47.605134 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" event={"ID":"02000744-cf65-4f6e-8f67-40b30f0d9cc3","Type":"ContainerStarted","Data":"aae2cbd2625fde8187ae0d5947702407d02347a490006d69087644c66ebd22f1"} Jul 23 18:00:47.611169 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:47.611125 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd" event={"ID":"4efa0791-cff4-4eef-9f3a-2a584c874ba5","Type":"ContainerStarted","Data":"82d48f8899bdb1e838dd7f9b6ecc92e02ee430a121b8b6892d41ba843d9bc463"} Jul 23 18:00:47.612379 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:47.612336 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" event={"ID":"570457cc-c6ec-4e45-8a1b-cb87156bd700","Type":"ContainerStarted","Data":"e7ecf24906c65d052fb1915f5946ec6e64c272a0cbf9ba2f14072b72e6286ff7"} Jul 23 18:00:48.484231 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:48.484172 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:48.486840 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:48.486814 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccdc7fc1-9212-4c7f-ad21-3916e5496d12-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2snwn\" (UID: \"ccdc7fc1-9212-4c7f-ad21-3916e5496d12\") " pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:48.701146 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:48.701109 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mq246\"" Jul 23 18:00:48.708817 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:48.708775 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2snwn" Jul 23 18:00:49.186770 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.186622 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2snwn"] Jul 23 18:00:49.361580 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.361534 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:49.413419 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.412131 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp"] Jul 23 18:00:49.414779 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.414691 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.418227 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.418100 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 23 18:00:49.418227 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.418140 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 23 18:00:49.418742 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.418522 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 18:00:49.418742 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.418603 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-blzwj\"" Jul 23 18:00:49.418742 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.418695 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 18:00:49.419970 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.419913 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 18:00:49.432688 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.432602 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp"] Jul 23 18:00:49.458455 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.458403 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:49.492792 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.492637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52885627-f5ff-42a6-b040-6e1e0c89934a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.492792 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.492673 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/52885627-f5ff-42a6-b040-6e1e0c89934a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.492792 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.492707 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52885627-f5ff-42a6-b040-6e1e0c89934a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.493371 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.492850 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r685\" (UniqueName: \"kubernetes.io/projected/52885627-f5ff-42a6-b040-6e1e0c89934a-kube-api-access-4r685\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.593961 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.593916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r685\" (UniqueName: \"kubernetes.io/projected/52885627-f5ff-42a6-b040-6e1e0c89934a-kube-api-access-4r685\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.594120 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.594087 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52885627-f5ff-42a6-b040-6e1e0c89934a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.594164 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.594131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/52885627-f5ff-42a6-b040-6e1e0c89934a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.594202 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.594173 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52885627-f5ff-42a6-b040-6e1e0c89934a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.594995 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.594944 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52885627-f5ff-42a6-b040-6e1e0c89934a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.597285 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.597245 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/52885627-f5ff-42a6-b040-6e1e0c89934a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.598310 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.598282 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52885627-f5ff-42a6-b040-6e1e0c89934a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.620204 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.620127 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r685\" (UniqueName: \"kubernetes.io/projected/52885627-f5ff-42a6-b040-6e1e0c89934a-kube-api-access-4r685\") pod \"prometheus-operator-5945b5cfd-7pzkp\" (UID: \"52885627-f5ff-42a6-b040-6e1e0c89934a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:49.627739 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.626890 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hzc9d" event={"ID":"84ca6f4e-056f-4d1f-93a4-2e910ef06f08","Type":"ContainerStarted","Data":"2774fd1e4fda235284753ef4ae038e6164906cff22b637a78cbf04d097597fb6"} Jul 23 18:00:49.627739 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.626963 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hzc9d" event={"ID":"84ca6f4e-056f-4d1f-93a4-2e910ef06f08","Type":"ContainerStarted","Data":"ad13a913ea8cb9b2e6b690ab0325b85f44e0708eb0cbc3dd38a768f205a1b5b6"} Jul 23 18:00:49.627739 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.627530 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hzc9d" Jul 23 18:00:49.633555 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.633514 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2snwn" event={"ID":"ccdc7fc1-9212-4c7f-ad21-3916e5496d12","Type":"ContainerStarted","Data":"9dd64c9a65d24db42b2048882789415853eeb43dcb035e4a7c3b63619c4a2e4a"} Jul 23 18:00:49.638942 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.638907 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" event={"ID":"02000744-cf65-4f6e-8f67-40b30f0d9cc3","Type":"ContainerStarted","Data":"753c89bf3ab6a61a265c94ac6057f95a822b3df2e7645604f9b31658948904c1"} Jul 23 18:00:49.639735 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.639711 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:00:49.642843 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.642797 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h62kj" event={"ID":"822c461b-5e7b-4abb-a090-edacc6b4392c","Type":"ContainerStarted","Data":"29484e472e27b3872b9d446648f85a5f7eb6bae867c236438838676cbae265d9"} Jul 23 18:00:49.646345 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.646308 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:49.660935 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.660577 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hzc9d" podStartSLOduration=33.993033227 podStartE2EDuration="38.660552965s" totalImagesPullingTime="4.667519738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:49.66031749 +0000 UTC m=+71.147809481" watchObservedRunningTime="2026-07-23 18:00:49.660552965 +0000 UTC m=+71.148044958" Jul 23 18:00:49.701274 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.699984 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" podStartSLOduration=3.699961553 podStartE2EDuration="3.699961553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:00:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:49.698515755 +0000 UTC m=+71.186007748" watchObservedRunningTime="2026-07-23 18:00:49.699961553 +0000 UTC m=+71.187453546" Jul 23 18:00:49.728252 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:49.728170 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" Jul 23 18:00:50.207034 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:50.206966 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h62kj" podStartSLOduration=66.293973908 podStartE2EDuration="1m11.206943835s" totalImagesPullingTime="4.912969927s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:49.732166785 +0000 UTC m=+71.219658800" watchObservedRunningTime="2026-07-23 18:00:50.206943835 +0000 UTC m=+71.694435823" Jul 23 18:00:50.207930 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:50.207909 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp"] Jul 23 18:00:50.863881 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:00:50.863846 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52885627_f5ff_42a6_b040_6e1e0c89934a.slice/crio-937db291dd230dd434f69bcbbbbccd1f0108660c8bf08067ce952b2043a9ec28 WatchSource:0}: Error finding container 937db291dd230dd434f69bcbbbbccd1f0108660c8bf08067ce952b2043a9ec28: Status 404 returned error can't find the container with id 937db291dd230dd434f69bcbbbbccd1f0108660c8bf08067ce952b2043a9ec28 Jul 23 18:00:51.653583 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:51.653494 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" event={"ID":"52885627-f5ff-42a6-b040-6e1e0c89934a","Type":"ContainerStarted","Data":"937db291dd230dd434f69bcbbbbccd1f0108660c8bf08067ce952b2043a9ec28"} Jul 23 18:00:51.656951 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:51.656917 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd" event={"ID":"4efa0791-cff4-4eef-9f3a-2a584c874ba5","Type":"ContainerStarted","Data":"c17f7b17da5348959d6b2825909e6dbdd7b06619825ef8a6a2f2fbcc2f8f28cb"} Jul 23 18:00:51.677900 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:51.676846 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-jhzbd" podStartSLOduration=3.179843018 podStartE2EDuration="5.676820612s" totalImagesPullingTime="2.496977594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:00:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:51.676029357 +0000 UTC m=+73.163521350" watchObservedRunningTime="2026-07-23 18:00:51.676820612 +0000 UTC m=+73.164312607" Jul 23 18:00:51.828984 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:51.828889 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:52.177919 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:52.177880 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:56.568172 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:56.568131 2562 patch_prober.go:28] interesting pod/image-registry-65f9d79445-h5c4b container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 18:00:56.568679 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:56.568224 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" podUID="0806e87e-c116-4fa0-b9ae-4f657a08b6b1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 18:00:57.118656 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:57.118617 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:57.286797 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:57.286759 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:57.420823 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:57.420717 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:00:57.684351 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:57.684264 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2snwn" event={"ID":"ccdc7fc1-9212-4c7f-ad21-3916e5496d12","Type":"ContainerStarted","Data":"fdf4c31ac5d8e36b51bd4213cf8740ef9fb0b60fbfa282774bcf16ab926c4982"} Jul 23 18:00:57.686798 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:57.686765 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" event={"ID":"52885627-f5ff-42a6-b040-6e1e0c89934a","Type":"ContainerStarted","Data":"3a916517d67a75508bca8050feba9ccd3d8ece2b4ded2665495008f974d42e8a"} Jul 23 18:00:57.686965 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:57.686824 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" event={"ID":"52885627-f5ff-42a6-b040-6e1e0c89934a","Type":"ContainerStarted","Data":"bf7077d6b134aec4257c08af37b1f267ab85940da105ac538929f9112a3ca44d"} Jul 23 18:00:57.689027 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:57.688994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" event={"ID":"570457cc-c6ec-4e45-8a1b-cb87156bd700","Type":"ContainerStarted","Data":"8354fbc4c97535a6350cdd70a4d69b86d2ea3f0cb46817ef36e1a98c52b022cf"} Jul 23 18:00:57.689335 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:57.689276 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:57.692294 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:57.692251 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" Jul 23 18:00:57.708978 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:57.708913 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-7pzkp" podStartSLOduration=2.897817295 podStartE2EDuration="8.708892342s" totalImagesPullingTime="5.811075047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:57.706823624 +0000 UTC m=+79.194315618" watchObservedRunningTime="2026-07-23 18:00:57.708892342 +0000 UTC m=+79.196384336" Jul 23 18:00:57.727056 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:57.726989 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6dcdb6c66d-hnlnl" podStartSLOduration=2.975947268 podStartE2EDuration="11.726964736s" totalImagesPullingTime="8.751017468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:00:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:00:57.725200173 +0000 UTC m=+79.212692185" watchObservedRunningTime="2026-07-23 18:00:57.726964736 +0000 UTC m=+79.214456730" Jul 23 18:00:59.653874 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:00:59.653821 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:00.187524 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.187489 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:00.232412 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.232386 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp"] Jul 23 18:01:00.236522 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.236498 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.241097 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.241069 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 23 18:01:00.241198 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.241076 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 23 18:01:00.241198 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.241178 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ktrsh\"" Jul 23 18:01:00.261264 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.261239 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp"] Jul 23 18:01:00.356475 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.356440 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ppjwf"] Jul 23 18:01:00.359158 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.359139 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.370855 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.370830 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 18:01:00.371454 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.371437 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5ghbr\"" Jul 23 18:01:00.371848 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.371834 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 18:01:00.371986 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.371976 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 18:01:00.393507 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.393474 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cb4db7-dec6-4353-b555-b56201b8d2e0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.393666 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.393531 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34cb4db7-dec6-4353-b555-b56201b8d2e0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.393666 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.393559 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dffjz\" (UniqueName: \"kubernetes.io/projected/34cb4db7-dec6-4353-b555-b56201b8d2e0-kube-api-access-dffjz\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.393666 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.393614 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34cb4db7-dec6-4353-b555-b56201b8d2e0-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.418481 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.418447 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:00.494878 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.494841 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-tls\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.494878 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.494877 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5lww\" (UniqueName: \"kubernetes.io/projected/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-kube-api-access-k5lww\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.495076 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.494905 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34cb4db7-dec6-4353-b555-b56201b8d2e0-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.495076 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.494966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-textfile\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.495076 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.495008 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cb4db7-dec6-4353-b555-b56201b8d2e0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.495076 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.495039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-wtmp\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.495266 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.495091 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-sys\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.495266 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:01:00.495177 2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 23 18:01:00.495266 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.495197 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-root\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.495374 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:01:00.495269 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34cb4db7-dec6-4353-b555-b56201b8d2e0-openshift-state-metrics-tls podName:34cb4db7-dec6-4353-b555-b56201b8d2e0 nodeName:}" failed. No retries permitted until 2026-07-23 18:01:00.995248257 +0000 UTC m=+82.482740242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/34cb4db7-dec6-4353-b555-b56201b8d2e0-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-ngtqp" (UID: "34cb4db7-dec6-4353-b555-b56201b8d2e0") : secret "openshift-state-metrics-tls" not found Jul 23 18:01:00.495374 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.495290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34cb4db7-dec6-4353-b555-b56201b8d2e0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.495374 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.495323 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-metrics-client-ca\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.495374 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.495348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.495531 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.495382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dffjz\" (UniqueName: \"kubernetes.io/projected/34cb4db7-dec6-4353-b555-b56201b8d2e0-kube-api-access-dffjz\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.495531 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.495409 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-accelerators-collector-config\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.495692 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.495669 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34cb4db7-dec6-4353-b555-b56201b8d2e0-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.497603 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.497577 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34cb4db7-dec6-4353-b555-b56201b8d2e0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.507196 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.507172 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dffjz\" (UniqueName: \"kubernetes.io/projected/34cb4db7-dec6-4353-b555-b56201b8d2e0-kube-api-access-dffjz\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:00.532977 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.532946 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:00.596393 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596358 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-wtmp\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.596559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596404 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-sys\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.596559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596421 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-root\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.596559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596447 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-metrics-client-ca\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.596559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.596559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596490 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-sys\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.596559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596500 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-accelerators-collector-config\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.596559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596498 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-root\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.596559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596550 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-wtmp\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.596893 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596573 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-tls\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.596893 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596601 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5lww\" (UniqueName: \"kubernetes.io/projected/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-kube-api-access-k5lww\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.596893 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-textfile\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.597011 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.596897 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-textfile\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.597127 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.597106 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-metrics-client-ca\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.597305 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.597288 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-accelerators-collector-config\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.598836 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.598817 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.598891 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.598876 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-node-exporter-tls\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.608237 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.608117 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5lww\" (UniqueName: \"kubernetes.io/projected/584ac4dd-5636-4cf6-9f5c-d461f9a3f53a-kube-api-access-k5lww\") pod \"node-exporter-ppjwf\" (UID: \"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a\") " pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.635671 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.635625 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:00.658555 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.658529 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hzc9d" Jul 23 18:01:00.668094 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.668075 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ppjwf" Jul 23 18:01:00.678430 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:01:00.678395 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod584ac4dd_5636_4cf6_9f5c_d461f9a3f53a.slice/crio-3be05a6303e1913dfc26f9735e0bcb160a5f9baf0f807aeb7664d1ff36b8b779 WatchSource:0}: Error finding container 3be05a6303e1913dfc26f9735e0bcb160a5f9baf0f807aeb7664d1ff36b8b779: Status 404 returned error can't find the container with id 3be05a6303e1913dfc26f9735e0bcb160a5f9baf0f807aeb7664d1ff36b8b779 Jul 23 18:01:00.702704 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.702661 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2snwn" event={"ID":"ccdc7fc1-9212-4c7f-ad21-3916e5496d12","Type":"ContainerStarted","Data":"b5637d6f95c28b70ee5c3a655cdf7365c6dc060bafb8bba5fbab6ef6fec8b2c4"} Jul 23 18:01:00.703957 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.703933 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ppjwf" event={"ID":"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a","Type":"ContainerStarted","Data":"3be05a6303e1913dfc26f9735e0bcb160a5f9baf0f807aeb7664d1ff36b8b779"} Jul 23 18:01:00.815926 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.815831 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:00.963845 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.963806 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:00.999979 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:00.999938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cb4db7-dec6-4353-b555-b56201b8d2e0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:01.002589 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:01.002560 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cb4db7-dec6-4353-b555-b56201b8d2e0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-ngtqp\" (UID: \"34cb4db7-dec6-4353-b555-b56201b8d2e0\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:01.062393 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:01.062356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:01.162727 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:01.162629 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" Jul 23 18:01:01.206573 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:01.206538 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:01.309426 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:01.309401 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp"] Jul 23 18:01:01.311242 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:01:01.311202 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34cb4db7_dec6_4353_b555_b56201b8d2e0.slice/crio-c203db2edd4511a7c9d2ced4cb6760c3df8994dcf2550bcaecb7c8e883207d06 WatchSource:0}: Error finding container c203db2edd4511a7c9d2ced4cb6760c3df8994dcf2550bcaecb7c8e883207d06: Status 404 returned error can't find the container with id c203db2edd4511a7c9d2ced4cb6760c3df8994dcf2550bcaecb7c8e883207d06 Jul 23 18:01:01.404721 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:01.404688 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:01.709765 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:01.709729 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" event={"ID":"34cb4db7-dec6-4353-b555-b56201b8d2e0","Type":"ContainerStarted","Data":"4ffed5977c236c4fd94426ab70c3fe964fc24d59f64ee1a412d1bf0678357bc3"} Jul 23 18:01:01.710154 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:01.709790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" event={"ID":"34cb4db7-dec6-4353-b555-b56201b8d2e0","Type":"ContainerStarted","Data":"bd952ba9f25b3e83685dab2fe1183d7c80fb17087c0a464737db711a5eacb868"} Jul 23 18:01:01.710154 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:01.709810 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" event={"ID":"34cb4db7-dec6-4353-b555-b56201b8d2e0","Type":"ContainerStarted","Data":"c203db2edd4511a7c9d2ced4cb6760c3df8994dcf2550bcaecb7c8e883207d06"} Jul 23 18:01:02.721388 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.721353 2562 generic.go:356] "Generic (PLEG): container finished" podID="584ac4dd-5636-4cf6-9f5c-d461f9a3f53a" containerID="124561322c9e03de5f483e0fe5c0542bcfa9938679ba3035b1e073ac82e69438" exitCode=0 Jul 23 18:01:02.721388 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.721408 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ppjwf" event={"ID":"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a","Type":"ContainerDied","Data":"124561322c9e03de5f483e0fe5c0542bcfa9938679ba3035b1e073ac82e69438"} Jul 23 18:01:02.809124 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.809087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:02.924273 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.924241 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:02.978563 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.978535 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5b678b85c5-5ksjg"] Jul 23 18:01:02.981511 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.981489 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:02.989093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.988833 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8l7ti3tt6qona\"" Jul 23 18:01:02.989093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.988889 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 23 18:01:02.989093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.988965 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 23 18:01:02.989093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.988889 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 23 18:01:02.994294 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.994272 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 23 18:01:02.994519 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.994332 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-5jjnf\"" Jul 23 18:01:02.996778 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:02.996585 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 23 18:01:03.052158 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.052059 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b678b85c5-5ksjg"] Jul 23 18:01:03.123513 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.123479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-tls\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.123683 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.123518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/390867ea-0d3a-48fc-b960-d961f286c476-metrics-client-ca\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.123683 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.123555 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx6dj\" (UniqueName: \"kubernetes.io/projected/390867ea-0d3a-48fc-b960-d961f286c476-kube-api-access-xx6dj\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.123683 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.123606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.123683 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.123640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.123877 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.123683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.123877 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.123716 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-grpc-tls\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.123877 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.123776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.148515 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.148478 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:03.224792 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.224752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-grpc-tls\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.224981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.224799 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.224981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.224864 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-tls\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.224981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.224890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/390867ea-0d3a-48fc-b960-d961f286c476-metrics-client-ca\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.224981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.224925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx6dj\" (UniqueName: \"kubernetes.io/projected/390867ea-0d3a-48fc-b960-d961f286c476-kube-api-access-xx6dj\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.224981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.224963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.226041 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.225003 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.226041 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.225057 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.226041 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.225751 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/390867ea-0d3a-48fc-b960-d961f286c476-metrics-client-ca\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.228028 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.228000 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.228125 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.228004 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.228664 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.228175 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-tls\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.228664 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.228583 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.228795 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.228695 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.229259 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.229207 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/390867ea-0d3a-48fc-b960-d961f286c476-secret-grpc-tls\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.253360 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.253336 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx6dj\" (UniqueName: \"kubernetes.io/projected/390867ea-0d3a-48fc-b960-d961f286c476-kube-api-access-xx6dj\") pod \"thanos-querier-5b678b85c5-5ksjg\" (UID: \"390867ea-0d3a-48fc-b960-d961f286c476\") " pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.295280 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.295244 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:03.461257 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.461134 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b678b85c5-5ksjg"] Jul 23 18:01:03.464705 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:01:03.464645 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390867ea_0d3a_48fc_b960_d961f286c476.slice/crio-4e4be86778afe57273e06734f5ca40cf2078f14c920afc983815466b4f9c28d2 WatchSource:0}: Error finding container 4e4be86778afe57273e06734f5ca40cf2078f14c920afc983815466b4f9c28d2: Status 404 returned error can't find the container with id 4e4be86778afe57273e06734f5ca40cf2078f14c920afc983815466b4f9c28d2 Jul 23 18:01:03.728528 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.728481 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" event={"ID":"34cb4db7-dec6-4353-b555-b56201b8d2e0","Type":"ContainerStarted","Data":"fa0929b793a0ab65d095825648c07b4fdee46691beb395caba66f518488863a0"} Jul 23 18:01:03.731141 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.731109 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ppjwf" event={"ID":"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a","Type":"ContainerStarted","Data":"27cf045cb460378644d57284d055c4007ce1285dd4239eb0a6cc4e30f12c8b35"} Jul 23 18:01:03.731297 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.731170 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ppjwf" event={"ID":"584ac4dd-5636-4cf6-9f5c-d461f9a3f53a","Type":"ContainerStarted","Data":"f46660097fec9c55e30868ef05ccf3c7f2e2bf12b7f6e303696b18707e29bd9f"} Jul 23 18:01:03.732511 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.732481 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" event={"ID":"390867ea-0d3a-48fc-b960-d961f286c476","Type":"ContainerStarted","Data":"4e4be86778afe57273e06734f5ca40cf2078f14c920afc983815466b4f9c28d2"} Jul 23 18:01:03.751423 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.751362 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-ngtqp" podStartSLOduration=2.25250363 podStartE2EDuration="3.751343998s" totalImagesPullingTime="1.498840368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:01:03.749623652 +0000 UTC m=+85.237115645" watchObservedRunningTime="2026-07-23 18:01:03.751343998 +0000 UTC m=+85.238835989" Jul 23 18:01:03.772251 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.772175 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ppjwf" podStartSLOduration=1.968776963 podStartE2EDuration="3.77215532s" totalImagesPullingTime="1.7271631s" totalInitContainerRuntime="76.215257ms" isStatefulPod=true podCreationTimestamp="2026-07-23 18:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:01:03.770485655 +0000 UTC m=+85.257977648" watchObservedRunningTime="2026-07-23 18:01:03.77215532 +0000 UTC m=+85.259647312" Jul 23 18:01:03.877107 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:03.877069 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:04.568386 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.568348 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn"] Jul 23 18:01:04.569616 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.569559 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:04.572945 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.572919 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" Jul 23 18:01:04.576067 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.576040 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 23 18:01:04.576367 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.576342 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-nwb6z\"" Jul 23 18:01:04.581406 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.581378 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn"] Jul 23 18:01:04.691569 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.691467 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:04.740207 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.740175 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcjrz\" (UniqueName: \"kubernetes.io/projected/821d3383-f3b4-4921-a42c-90d1ec4a0993-kube-api-access-hcjrz\") pod \"monitoring-plugin-7b7b7bc8bf-ck2nn\" (UID: \"821d3383-f3b4-4921-a42c-90d1ec4a0993\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" Jul 23 18:01:04.740676 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.740287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/821d3383-f3b4-4921-a42c-90d1ec4a0993-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-ck2nn\" (UID: \"821d3383-f3b4-4921-a42c-90d1ec4a0993\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" Jul 23 18:01:04.842307 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.841682 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/821d3383-f3b4-4921-a42c-90d1ec4a0993-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-ck2nn\" (UID: \"821d3383-f3b4-4921-a42c-90d1ec4a0993\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" Jul 23 18:01:04.842307 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.841775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcjrz\" (UniqueName: \"kubernetes.io/projected/821d3383-f3b4-4921-a42c-90d1ec4a0993-kube-api-access-hcjrz\") pod \"monitoring-plugin-7b7b7bc8bf-ck2nn\" (UID: \"821d3383-f3b4-4921-a42c-90d1ec4a0993\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" Jul 23 18:01:04.847235 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.847187 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/821d3383-f3b4-4921-a42c-90d1ec4a0993-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-ck2nn\" (UID: \"821d3383-f3b4-4921-a42c-90d1ec4a0993\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" Jul 23 18:01:04.851457 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.851430 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcjrz\" (UniqueName: \"kubernetes.io/projected/821d3383-f3b4-4921-a42c-90d1ec4a0993-kube-api-access-hcjrz\") pod \"monitoring-plugin-7b7b7bc8bf-ck2nn\" (UID: \"821d3383-f3b4-4921-a42c-90d1ec4a0993\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" Jul 23 18:01:04.871499 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.871460 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:04.888528 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:04.888494 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" Jul 23 18:01:05.968588 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:05.968549 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:06.109623 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:06.109565 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:06.190393 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:06.190356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:06.568891 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:06.568859 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:01:06.957494 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:06.957463 2562 patch_prober.go:28] interesting pod/image-registry-65749cb4f-4xvjg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 18:01:06.957635 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:06.957520 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" podUID="02000744-cf65-4f6e-8f67-40b30f0d9cc3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 18:01:06.987961 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:06.987847 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn"] Jul 23 18:01:06.990434 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:01:06.990410 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod821d3383_f3b4_4921_a42c_90d1ec4a0993.slice/crio-7c078e7f93c908ffde4aef7cf7338737b668378ef1621491f44c15059b7e7763 WatchSource:0}: Error finding container 7c078e7f93c908ffde4aef7cf7338737b668378ef1621491f44c15059b7e7763: Status 404 returned error can't find the container with id 7c078e7f93c908ffde4aef7cf7338737b668378ef1621491f44c15059b7e7763 Jul 23 18:01:07.755406 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:07.755362 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2snwn" event={"ID":"ccdc7fc1-9212-4c7f-ad21-3916e5496d12","Type":"ContainerStarted","Data":"4190319c8810e0b3c94dcadfd5204612686fb1ee29b4db7c2d47dd3540f2b74a"} Jul 23 18:01:07.757371 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:07.757342 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" event={"ID":"390867ea-0d3a-48fc-b960-d961f286c476","Type":"ContainerStarted","Data":"9aa092796f326bf611af8ea38c45ed5cb1df65fedc4b4df531d8f9ba8186ac87"} Jul 23 18:01:07.757503 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:07.757421 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" event={"ID":"390867ea-0d3a-48fc-b960-d961f286c476","Type":"ContainerStarted","Data":"186f82ae1a756c3100fc20aa1192bef11293dc2a613e9f93481684f13555f709"} Jul 23 18:01:07.757503 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:07.757464 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" event={"ID":"390867ea-0d3a-48fc-b960-d961f286c476","Type":"ContainerStarted","Data":"a34b4a1d64eb79966c59fa924e870b997c33f45e51aba9dc78533d710e3ce132"} Jul 23 18:01:07.758412 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:07.758391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" event={"ID":"821d3383-f3b4-4921-a42c-90d1ec4a0993","Type":"ContainerStarted","Data":"7c078e7f93c908ffde4aef7cf7338737b668378ef1621491f44c15059b7e7763"} Jul 23 18:01:07.772978 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:07.772924 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2snwn" podStartSLOduration=34.235744469 podStartE2EDuration="51.772904594s" totalImagesPullingTime="17.537160125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:00:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:01:07.772689678 +0000 UTC m=+89.260181670" watchObservedRunningTime="2026-07-23 18:01:07.772904594 +0000 UTC m=+89.260396590" Jul 23 18:01:08.770059 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:08.770010 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" event={"ID":"821d3383-f3b4-4921-a42c-90d1ec4a0993","Type":"ContainerStarted","Data":"b5e2dbfe2364aa4157d6e2aed8655b77c27a37106569e84a63b104af2a4ea165"} Jul 23 18:01:08.770889 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:08.770408 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" Jul 23 18:01:08.775570 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:08.775541 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" Jul 23 18:01:08.787286 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:08.787230 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ck2nn" podStartSLOduration=3.354428254 podStartE2EDuration="4.787191105s" totalImagesPullingTime="1.432762851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:01:08.786969232 +0000 UTC m=+90.274461243" watchObservedRunningTime="2026-07-23 18:01:08.787191105 +0000 UTC m=+90.274683098" Jul 23 18:01:09.200145 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:09.200114 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:09.301500 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:09.301404 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:09.776816 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:09.776774 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" event={"ID":"390867ea-0d3a-48fc-b960-d961f286c476","Type":"ContainerStarted","Data":"9321830ffcc1fcfc93279382e84f28ca81e734264bb7acd3b75854c6d3abe9ab"} Jul 23 18:01:09.777205 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:09.776857 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" event={"ID":"390867ea-0d3a-48fc-b960-d961f286c476","Type":"ContainerStarted","Data":"4692da6c2169f08aceb9032a27bd21fc6a3c7dc24a0060533a4a01168c456bb3"} Jul 23 18:01:09.777205 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:09.776900 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" event={"ID":"390867ea-0d3a-48fc-b960-d961f286c476","Type":"ContainerStarted","Data":"7160116a4cfcdb3e90f3dd01c5e64af957c325d9b9c9bd245fbc13568d8911a3"} Jul 23 18:01:09.812543 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:09.812490 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" podStartSLOduration=2.558999603 podStartE2EDuration="7.812470822s" totalImagesPullingTime="5.253471219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:01:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:01:09.810363689 +0000 UTC m=+91.297855682" watchObservedRunningTime="2026-07-23 18:01:09.812470822 +0000 UTC m=+91.299962814" Jul 23 18:01:09.854645 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:09.854601 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:10.211994 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:10.211959 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:10.286428 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:10.286395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:10.409769 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:10.409728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:10.511101 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:10.510994 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:10.652258 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:10.652203 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65749cb4f-4xvjg" Jul 23 18:01:10.781430 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:10.781346 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:11.189777 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:11.189692 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:11.299357 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:11.299331 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:11.592333 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:11.592293 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" podUID="0806e87e-c116-4fa0-b9ae-4f657a08b6b1" containerName="registry" containerID="cri-o://5c7136a76500b01de6e52e13061428671af361cf5c428930c5296afcbccdfe17" gracePeriod=30 Jul 23 18:01:11.786811 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:11.786772 2562 generic.go:356] "Generic (PLEG): container finished" podID="0806e87e-c116-4fa0-b9ae-4f657a08b6b1" containerID="5c7136a76500b01de6e52e13061428671af361cf5c428930c5296afcbccdfe17" exitCode=0 Jul 23 18:01:11.787155 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:11.786845 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" event={"ID":"0806e87e-c116-4fa0-b9ae-4f657a08b6b1","Type":"ContainerDied","Data":"5c7136a76500b01de6e52e13061428671af361cf5c428930c5296afcbccdfe17"} Jul 23 18:01:11.792563 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:11.792540 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b678b85c5-5ksjg" Jul 23 18:01:11.846467 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:11.846399 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:01:12.005741 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.005702 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-ca-trust-extracted\") pod \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " Jul 23 18:01:12.005980 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.005963 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-installation-pull-secrets\") pod \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " Jul 23 18:01:12.006127 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.006113 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-certificates\") pod \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " Jul 23 18:01:12.006638 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.006620 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-image-registry-private-configuration\") pod \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " Jul 23 18:01:12.007083 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.007061 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-bound-sa-token\") pod \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " Jul 23 18:01:12.007192 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.006570 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0806e87e-c116-4fa0-b9ae-4f657a08b6b1" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:01:12.007192 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.007145 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22dst\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-kube-api-access-22dst\") pod \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " Jul 23 18:01:12.007192 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.007178 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-trusted-ca\") pod \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " Jul 23 18:01:12.007338 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.007256 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls\") pod \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\" (UID: \"0806e87e-c116-4fa0-b9ae-4f657a08b6b1\") " Jul 23 18:01:12.007496 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.007479 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-certificates\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:01:12.009483 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.008102 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0806e87e-c116-4fa0-b9ae-4f657a08b6b1" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:01:12.009775 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.009719 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0806e87e-c116-4fa0-b9ae-4f657a08b6b1" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:01:12.010087 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.010059 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0806e87e-c116-4fa0-b9ae-4f657a08b6b1" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:01:12.010168 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.010142 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0806e87e-c116-4fa0-b9ae-4f657a08b6b1" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:01:12.010242 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.010182 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-kube-api-access-22dst" (OuterVolumeSpecName: "kube-api-access-22dst") pod "0806e87e-c116-4fa0-b9ae-4f657a08b6b1" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1"). InnerVolumeSpecName "kube-api-access-22dst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:01:12.010310 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.010291 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0806e87e-c116-4fa0-b9ae-4f657a08b6b1" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:01:12.015664 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.015634 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0806e87e-c116-4fa0-b9ae-4f657a08b6b1" (UID: "0806e87e-c116-4fa0-b9ae-4f657a08b6b1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:01:12.108826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.108729 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-registry-tls\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:01:12.108826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.108768 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-ca-trust-extracted\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:01:12.108826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.108784 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-installation-pull-secrets\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:01:12.108826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.108799 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-image-registry-private-configuration\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:01:12.108826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.108818 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-bound-sa-token\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:01:12.109086 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.108832 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22dst\" (UniqueName: \"kubernetes.io/projected/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-kube-api-access-22dst\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:01:12.109086 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.108847 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0806e87e-c116-4fa0-b9ae-4f657a08b6b1-trusted-ca\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:01:12.793025 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.792992 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" Jul 23 18:01:12.793547 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.793024 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f9d79445-h5c4b" event={"ID":"0806e87e-c116-4fa0-b9ae-4f657a08b6b1","Type":"ContainerDied","Data":"453436301db69e7f01062e15d3d30454f7b595a91ef8782aeab838648c5fccd4"} Jul 23 18:01:12.793547 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.793087 2562 scope.go:117] "RemoveContainer" containerID="5c7136a76500b01de6e52e13061428671af361cf5c428930c5296afcbccdfe17" Jul 23 18:01:12.830267 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.830232 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65f9d79445-h5c4b"] Jul 23 18:01:12.834484 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:12.834457 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65f9d79445-h5c4b"] Jul 23 18:01:13.129735 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:13.129654 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0806e87e-c116-4fa0-b9ae-4f657a08b6b1" path="/var/lib/kubelet/pods/0806e87e-c116-4fa0-b9ae-4f657a08b6b1/volumes" Jul 23 18:01:15.178485 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:15.178426 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:15.272998 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:15.272965 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:15.596181 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:15.596150 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l8nhz" Jul 23 18:01:15.672813 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:15.672762 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:15.947087 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:15.947051 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:16.060560 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:16.060521 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:16.195491 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:16.195451 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:16.295759 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:16.295650 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:16.361172 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:16.361137 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:18.338739 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:18.338658 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:18.441060 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:18.441029 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:22.575528 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:22.575503 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hzc9d_84ca6f4e-056f-4d1f-93a4-2e910ef06f08/dns/0.log" Jul 23 18:01:22.776081 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:22.776048 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hzc9d_84ca6f4e-056f-4d1f-93a4-2e910ef06f08/kube-rbac-proxy/0.log" Jul 23 18:01:23.775936 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:23.775907 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngf5j_b673b420-6fcb-4021-8ea7-1dce6101f1ad/dns-node-resolver/0.log" Jul 23 18:01:24.184628 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:24.184554 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fsxnc_0833f914-e35c-4a6d-8a60-0e99269bee04/serve-healthcheck-canary/0.log" Jul 23 18:01:25.322428 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:25.322395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:25.386170 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:25.386138 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:25.577272 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:25.577173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:25.679836 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:25.679797 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:25.743876 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:25.743837 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:30.094035 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:30.093998 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:30.164082 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:30.164028 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:33.483388 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:33.483357 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:33.552855 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:33.552820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:33.878202 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:33.878123 2562 generic.go:356] "Generic (PLEG): container finished" podID="e422aa7b-a5f0-44d6-8819-d6619b6b455b" containerID="74e665d7cf6750e1b8d6a6b3a164edc7ef89c23ee815fa8b1f5cf60017f5bb81" exitCode=0 Jul 23 18:01:33.878202 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:33.878160 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" event={"ID":"e422aa7b-a5f0-44d6-8819-d6619b6b455b","Type":"ContainerDied","Data":"74e665d7cf6750e1b8d6a6b3a164edc7ef89c23ee815fa8b1f5cf60017f5bb81"} Jul 23 18:01:33.878556 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:33.878540 2562 scope.go:117] "RemoveContainer" containerID="74e665d7cf6750e1b8d6a6b3a164edc7ef89c23ee815fa8b1f5cf60017f5bb81" Jul 23 18:01:34.884359 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:34.884324 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-62b6j" event={"ID":"e422aa7b-a5f0-44d6-8819-d6619b6b455b","Type":"ContainerStarted","Data":"ddaa62a42479d842fc506f705b6dd09b72fc7e7d1c840a553608bf930a1cccc9"} Jul 23 18:01:45.203443 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:45.203400 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:45.303126 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:45.303083 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:45.927756 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:45.927720 2562 generic.go:356] "Generic (PLEG): container finished" podID="8152bbb3-da20-460a-b6e7-2a5aa61bee78" containerID="f90a543860ff84f72d3b84210675f5df2324ab8291e871613daaffede09fbf9e" exitCode=0 Jul 23 18:01:45.927950 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:45.927801 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" event={"ID":"8152bbb3-da20-460a-b6e7-2a5aa61bee78","Type":"ContainerDied","Data":"f90a543860ff84f72d3b84210675f5df2324ab8291e871613daaffede09fbf9e"} Jul 23 18:01:45.928192 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:45.928177 2562 scope.go:117] "RemoveContainer" containerID="f90a543860ff84f72d3b84210675f5df2324ab8291e871613daaffede09fbf9e" Jul 23 18:01:46.933886 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:46.933850 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-wgqrh" event={"ID":"8152bbb3-da20-460a-b6e7-2a5aa61bee78","Type":"ContainerStarted","Data":"64f9c257cfaf8a217977f626443f69647a233f57b601b0c68e01a2c4042c5c22"} Jul 23 18:01:58.115324 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:58.115284 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:01:58.197275 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:01:58.197201 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:19.196444 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:19.196404 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:19.856170 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:19.856104 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:24.309244 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.309184 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6659667f4d-2gnhv"] Jul 23 18:02:24.310204 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.310179 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0806e87e-c116-4fa0-b9ae-4f657a08b6b1" containerName="registry" Jul 23 18:02:24.310366 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.310354 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0806e87e-c116-4fa0-b9ae-4f657a08b6b1" containerName="registry" Jul 23 18:02:24.310535 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.310524 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0806e87e-c116-4fa0-b9ae-4f657a08b6b1" containerName="registry" Jul 23 18:02:24.314047 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.314025 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.318390 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.317035 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 23 18:02:24.318390 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.317377 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 23 18:02:24.318390 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.317605 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-qh78r\"" Jul 23 18:02:24.318390 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.318390 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 23 18:02:24.318668 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.318633 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 23 18:02:24.318908 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.318886 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 23 18:02:24.324540 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.324512 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 23 18:02:24.328452 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.328432 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6659667f4d-2gnhv"] Jul 23 18:02:24.487832 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.487791 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.487832 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.487832 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8tcp\" (UniqueName: \"kubernetes.io/projected/1dff4e18-e2f0-4157-b487-6dcb65870f0b-kube-api-access-v8tcp\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.488050 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.487869 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dff4e18-e2f0-4157-b487-6dcb65870f0b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.488050 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.487940 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-telemeter-client-tls\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.488050 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.487963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dff4e18-e2f0-4157-b487-6dcb65870f0b-serving-certs-ca-bundle\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.488050 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.487983 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-secret-telemeter-client\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.488050 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.488010 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dff4e18-e2f0-4157-b487-6dcb65870f0b-metrics-client-ca\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.488204 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.488049 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-federate-client-tls\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.589049 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.588933 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.589049 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.588994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8tcp\" (UniqueName: \"kubernetes.io/projected/1dff4e18-e2f0-4157-b487-6dcb65870f0b-kube-api-access-v8tcp\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.589324 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.589054 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dff4e18-e2f0-4157-b487-6dcb65870f0b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.589324 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.589094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-telemeter-client-tls\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.589324 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.589125 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dff4e18-e2f0-4157-b487-6dcb65870f0b-serving-certs-ca-bundle\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.589324 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.589156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-secret-telemeter-client\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.589324 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.589183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dff4e18-e2f0-4157-b487-6dcb65870f0b-metrics-client-ca\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.589324 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.589235 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-federate-client-tls\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.590319 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.590285 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dff4e18-e2f0-4157-b487-6dcb65870f0b-metrics-client-ca\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.590525 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.590475 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dff4e18-e2f0-4157-b487-6dcb65870f0b-serving-certs-ca-bundle\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.590647 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.590556 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dff4e18-e2f0-4157-b487-6dcb65870f0b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.592228 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.592183 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-federate-client-tls\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.592472 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.592447 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.593053 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.593034 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-telemeter-client-tls\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.593260 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.593242 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1dff4e18-e2f0-4157-b487-6dcb65870f0b-secret-telemeter-client\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.599562 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.599527 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8tcp\" (UniqueName: \"kubernetes.io/projected/1dff4e18-e2f0-4157-b487-6dcb65870f0b-kube-api-access-v8tcp\") pod \"telemeter-client-6659667f4d-2gnhv\" (UID: \"1dff4e18-e2f0-4157-b487-6dcb65870f0b\") " pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.625142 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.625102 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" Jul 23 18:02:24.782699 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:24.782604 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6659667f4d-2gnhv"] Jul 23 18:02:24.787938 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:02:24.787911 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dff4e18_e2f0_4157_b487_6dcb65870f0b.slice/crio-c37b5cee26214a2a31c8a8497369f714382eaca683ac6c99b5f10339f065e46f WatchSource:0}: Error finding container c37b5cee26214a2a31c8a8497369f714382eaca683ac6c99b5f10339f065e46f: Status 404 returned error can't find the container with id c37b5cee26214a2a31c8a8497369f714382eaca683ac6c99b5f10339f065e46f Jul 23 18:02:25.077540 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:25.077502 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" event={"ID":"1dff4e18-e2f0-4157-b487-6dcb65870f0b","Type":"ContainerStarted","Data":"c37b5cee26214a2a31c8a8497369f714382eaca683ac6c99b5f10339f065e46f"} Jul 23 18:02:25.951879 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:25.951840 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:26.286482 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:26.286399 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:29.097525 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:29.097486 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" event={"ID":"1dff4e18-e2f0-4157-b487-6dcb65870f0b","Type":"ContainerStarted","Data":"499c3aea38f3bfdd0cbf14f274010e937b8a9711a450d2972217d04d67736fea"} Jul 23 18:02:30.102947 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:30.102901 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" event={"ID":"1dff4e18-e2f0-4157-b487-6dcb65870f0b","Type":"ContainerStarted","Data":"080a0c3bd05c5d93fd82c818c7c429e0e56f4412546ae778041645f997d422ac"} Jul 23 18:02:30.103373 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:30.103010 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" event={"ID":"1dff4e18-e2f0-4157-b487-6dcb65870f0b","Type":"ContainerStarted","Data":"76edfc4e74bb3e479672832e1da657763e954cf3634c718f5469a56a546f6127"} Jul 23 18:02:30.134348 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:30.134300 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6659667f4d-2gnhv" podStartSLOduration=1.461509607 podStartE2EDuration="6.134283324s" totalImagesPullingTime="4.672773717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:02:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:02:30.133462928 +0000 UTC m=+171.620954939" watchObservedRunningTime="2026-07-23 18:02:30.134283324 +0000 UTC m=+171.621775314" Jul 23 18:02:30.975733 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:30.975627 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:31.078365 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:31.078333 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:31.618655 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:31.618614 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:31.719735 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:31.719701 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:37.559589 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:37.559554 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:41.398610 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:41.398575 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:42.259876 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:42.259833 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:43.112637 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:43.112599 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:02:52.149262 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:02:52.149199 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:03:34.531629 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:03:34.531595 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:03:34.672108 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:03:34.672064 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:03:36.679371 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:03:36.679331 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:03:36.757662 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:03:36.757625 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:03:39.248489 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:03:39.248450 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:03:46.613017 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:03:46.612982 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:03:46.690387 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:03:46.690353 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:03:47.689682 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:03:47.689638 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:03:47.790858 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:03:47.790823 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:03:47.859495 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:03:47.859456 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:04:00.234385 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:04:00.234347 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:04:00.370013 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:04:00.369976 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:04:07.134354 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:04:07.134317 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:04:07.217278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:04:07.217241 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:04:18.932507 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:04:18.932464 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:04:19.065190 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:04:19.065158 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:04:38.913697 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:04:38.913659 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:04:38.916462 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:04:38.916433 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:04:38.924644 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:04:38.924622 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 18:05:57.335509 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:05:57.335473 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:03.233426 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:03.233383 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:04.452620 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:04.452571 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:05.612916 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.612876 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l"] Jul 23 18:06:05.615178 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.615160 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l" Jul 23 18:06:05.617790 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.617762 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qkd4h\"" Jul 23 18:06:05.617790 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.617771 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 23 18:06:05.617957 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.617851 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 23 18:06:05.640819 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.640788 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l"] Jul 23 18:06:05.690530 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.690500 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw674\" (UniqueName: \"kubernetes.io/projected/d6881c3b-8014-4ac9-aa11-59a8720e4991-kube-api-access-dw674\") pod \"cert-manager-operator-controller-manager-55c444bf86-ph68l\" (UID: \"d6881c3b-8014-4ac9-aa11-59a8720e4991\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l" Jul 23 18:06:05.690682 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.690553 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6881c3b-8014-4ac9-aa11-59a8720e4991-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ph68l\" (UID: \"d6881c3b-8014-4ac9-aa11-59a8720e4991\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l" Jul 23 18:06:05.791513 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.791471 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6881c3b-8014-4ac9-aa11-59a8720e4991-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ph68l\" (UID: \"d6881c3b-8014-4ac9-aa11-59a8720e4991\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l" Jul 23 18:06:05.791699 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.791558 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw674\" (UniqueName: \"kubernetes.io/projected/d6881c3b-8014-4ac9-aa11-59a8720e4991-kube-api-access-dw674\") pod \"cert-manager-operator-controller-manager-55c444bf86-ph68l\" (UID: \"d6881c3b-8014-4ac9-aa11-59a8720e4991\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l" Jul 23 18:06:05.791917 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.791891 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6881c3b-8014-4ac9-aa11-59a8720e4991-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ph68l\" (UID: \"d6881c3b-8014-4ac9-aa11-59a8720e4991\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l" Jul 23 18:06:05.801737 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.801715 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw674\" (UniqueName: \"kubernetes.io/projected/d6881c3b-8014-4ac9-aa11-59a8720e4991-kube-api-access-dw674\") pod \"cert-manager-operator-controller-manager-55c444bf86-ph68l\" (UID: \"d6881c3b-8014-4ac9-aa11-59a8720e4991\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l" Jul 23 18:06:05.923777 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:05.923679 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l" Jul 23 18:06:06.070084 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:06.070050 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l"] Jul 23 18:06:06.076563 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:06:06.076530 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6881c3b_8014_4ac9_aa11_59a8720e4991.slice/crio-6d562bcb86b2fe9c633aa74f52a6e4601fb2710b92d536514f1422920fd8152c WatchSource:0}: Error finding container 6d562bcb86b2fe9c633aa74f52a6e4601fb2710b92d536514f1422920fd8152c: Status 404 returned error can't find the container with id 6d562bcb86b2fe9c633aa74f52a6e4601fb2710b92d536514f1422920fd8152c Jul 23 18:06:06.079255 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:06.079238 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 18:06:06.939553 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:06.939516 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l" event={"ID":"d6881c3b-8014-4ac9-aa11-59a8720e4991","Type":"ContainerStarted","Data":"6d562bcb86b2fe9c633aa74f52a6e4601fb2710b92d536514f1422920fd8152c"} Jul 23 18:06:07.063906 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:07.063869 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:07.169162 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:07.169125 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:07.506343 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:07.506305 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:07.607191 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:07.607149 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:08.244223 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:08.244188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:08.362221 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:08.362173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:09.757742 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:09.757700 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:09.863137 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:09.863101 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:09.952530 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:09.952493 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l" event={"ID":"d6881c3b-8014-4ac9-aa11-59a8720e4991","Type":"ContainerStarted","Data":"35edc6604cfb0b04597793be990b4b281032cbacec160e902bdb6a7cf70c1d5e"} Jul 23 18:06:10.477081 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:10.477043 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:10.625603 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:10.625562 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:12.171376 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.171328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:12.338621 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.338549 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ph68l" podStartSLOduration=4.422841426 podStartE2EDuration="7.338529858s" totalImagesPullingTime="2.915688432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:10.0005969 +0000 UTC m=+391.488088893" watchObservedRunningTime="2026-07-23 18:06:12.338529858 +0000 UTC m=+393.826021850" Jul 23 18:06:12.339876 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.339855 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dlczz"] Jul 23 18:06:12.343128 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.343110 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" Jul 23 18:06:12.346633 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.346613 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 23 18:06:12.347332 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.347314 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 23 18:06:12.347607 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.347588 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-x776q\"" Jul 23 18:06:12.351468 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.351448 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dlczz"] Jul 23 18:06:12.448504 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.448469 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da1aea48-c7ee-453b-8a1d-c402900c4614-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dlczz\" (UID: \"da1aea48-c7ee-453b-8a1d-c402900c4614\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" Jul 23 18:06:12.448672 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.448573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn8ng\" (UniqueName: \"kubernetes.io/projected/da1aea48-c7ee-453b-8a1d-c402900c4614-kube-api-access-xn8ng\") pod \"cert-manager-webhook-6486ffd997-dlczz\" (UID: \"da1aea48-c7ee-453b-8a1d-c402900c4614\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" Jul 23 18:06:12.549015 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.548975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da1aea48-c7ee-453b-8a1d-c402900c4614-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dlczz\" (UID: \"da1aea48-c7ee-453b-8a1d-c402900c4614\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" Jul 23 18:06:12.549201 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.549055 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn8ng\" (UniqueName: \"kubernetes.io/projected/da1aea48-c7ee-453b-8a1d-c402900c4614-kube-api-access-xn8ng\") pod \"cert-manager-webhook-6486ffd997-dlczz\" (UID: \"da1aea48-c7ee-453b-8a1d-c402900c4614\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" Jul 23 18:06:12.557768 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.557732 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:12.561692 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.561672 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da1aea48-c7ee-453b-8a1d-c402900c4614-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dlczz\" (UID: \"da1aea48-c7ee-453b-8a1d-c402900c4614\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" Jul 23 18:06:12.562142 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.562118 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn8ng\" (UniqueName: \"kubernetes.io/projected/da1aea48-c7ee-453b-8a1d-c402900c4614-kube-api-access-xn8ng\") pod \"cert-manager-webhook-6486ffd997-dlczz\" (UID: \"da1aea48-c7ee-453b-8a1d-c402900c4614\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" Jul 23 18:06:12.661010 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.660977 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" Jul 23 18:06:12.795810 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.795787 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dlczz"] Jul 23 18:06:12.797724 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:06:12.797698 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda1aea48_c7ee_453b_8a1d_c402900c4614.slice/crio-8a7c8977fafa43d04c524095bb95e25ce6f6d911866576e46c483b5176442678 WatchSource:0}: Error finding container 8a7c8977fafa43d04c524095bb95e25ce6f6d911866576e46c483b5176442678: Status 404 returned error can't find the container with id 8a7c8977fafa43d04c524095bb95e25ce6f6d911866576e46c483b5176442678 Jul 23 18:06:12.965797 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:12.965712 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" event={"ID":"da1aea48-c7ee-453b-8a1d-c402900c4614","Type":"ContainerStarted","Data":"8a7c8977fafa43d04c524095bb95e25ce6f6d911866576e46c483b5176442678"} Jul 23 18:06:14.021142 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:14.021103 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:14.127308 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:14.127271 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:15.980977 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:15.980946 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" event={"ID":"da1aea48-c7ee-453b-8a1d-c402900c4614","Type":"ContainerStarted","Data":"b6219b5a202cf3a94d8369f94734786397d7777285ae72d6ae550b7d100f50db"} Jul 23 18:06:15.981337 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:15.981009 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" Jul 23 18:06:16.002264 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:16.002197 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" podStartSLOduration=0.915922211 podStartE2EDuration="4.002182423s" totalImagesPullingTime="3.086260212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:16.000873983 +0000 UTC m=+397.488365998" watchObservedRunningTime="2026-07-23 18:06:16.002182423 +0000 UTC m=+397.489674415" Jul 23 18:06:20.721311 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:20.721272 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:21.532103 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:21.532065 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:21.665855 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:21.665820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:21.892474 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:21.892389 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fmdxr"] Jul 23 18:06:21.894893 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:21.894876 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fmdxr" Jul 23 18:06:21.897449 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:21.897427 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-q8zzk\"" Jul 23 18:06:21.905090 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:21.905068 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fmdxr"] Jul 23 18:06:21.928406 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:21.928374 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt85b\" (UniqueName: \"kubernetes.io/projected/5fe90316-7b4f-42cb-aeee-1e3e53ae8012-kube-api-access-gt85b\") pod \"cert-manager-55b4d954d5-fmdxr\" (UID: \"5fe90316-7b4f-42cb-aeee-1e3e53ae8012\") " pod="cert-manager/cert-manager-55b4d954d5-fmdxr" Jul 23 18:06:21.928571 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:21.928429 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5fe90316-7b4f-42cb-aeee-1e3e53ae8012-bound-sa-token\") pod \"cert-manager-55b4d954d5-fmdxr\" (UID: \"5fe90316-7b4f-42cb-aeee-1e3e53ae8012\") " pod="cert-manager/cert-manager-55b4d954d5-fmdxr" Jul 23 18:06:21.987932 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:21.987891 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dlczz" Jul 23 18:06:22.029441 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.029396 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt85b\" (UniqueName: \"kubernetes.io/projected/5fe90316-7b4f-42cb-aeee-1e3e53ae8012-kube-api-access-gt85b\") pod \"cert-manager-55b4d954d5-fmdxr\" (UID: \"5fe90316-7b4f-42cb-aeee-1e3e53ae8012\") " pod="cert-manager/cert-manager-55b4d954d5-fmdxr" Jul 23 18:06:22.029624 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.029459 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5fe90316-7b4f-42cb-aeee-1e3e53ae8012-bound-sa-token\") pod \"cert-manager-55b4d954d5-fmdxr\" (UID: \"5fe90316-7b4f-42cb-aeee-1e3e53ae8012\") " pod="cert-manager/cert-manager-55b4d954d5-fmdxr" Jul 23 18:06:22.040636 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.040602 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5fe90316-7b4f-42cb-aeee-1e3e53ae8012-bound-sa-token\") pod \"cert-manager-55b4d954d5-fmdxr\" (UID: \"5fe90316-7b4f-42cb-aeee-1e3e53ae8012\") " pod="cert-manager/cert-manager-55b4d954d5-fmdxr" Jul 23 18:06:22.040830 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.040806 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt85b\" (UniqueName: \"kubernetes.io/projected/5fe90316-7b4f-42cb-aeee-1e3e53ae8012-kube-api-access-gt85b\") pod \"cert-manager-55b4d954d5-fmdxr\" (UID: \"5fe90316-7b4f-42cb-aeee-1e3e53ae8012\") " pod="cert-manager/cert-manager-55b4d954d5-fmdxr" Jul 23 18:06:22.204556 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.204523 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fmdxr" Jul 23 18:06:22.336045 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.335846 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fmdxr"] Jul 23 18:06:22.338889 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:06:22.338861 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fe90316_7b4f_42cb_aeee_1e3e53ae8012.slice/crio-bcd3bb13f309f64d049d599638cf32df2a6e16c4e42a2df5f44c4de910efba04 WatchSource:0}: Error finding container bcd3bb13f309f64d049d599638cf32df2a6e16c4e42a2df5f44c4de910efba04: Status 404 returned error can't find the container with id bcd3bb13f309f64d049d599638cf32df2a6e16c4e42a2df5f44c4de910efba04 Jul 23 18:06:22.904223 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.904172 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x"] Jul 23 18:06:22.906529 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.906512 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x" Jul 23 18:06:22.909157 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.909138 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 18:06:22.909157 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.909145 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 18:06:22.909804 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.909789 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-jmvv2\"" Jul 23 18:06:22.916337 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.916316 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x"] Jul 23 18:06:22.935481 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.935454 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2de6c0f-4a95-4100-a495-47e47c7d1575-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7tt9x\" (UID: \"d2de6c0f-4a95-4100-a495-47e47c7d1575\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x" Jul 23 18:06:22.935600 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:22.935492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfjz2\" (UniqueName: \"kubernetes.io/projected/d2de6c0f-4a95-4100-a495-47e47c7d1575-kube-api-access-kfjz2\") pod \"openshift-lws-operator-fd8ccff4c-7tt9x\" (UID: \"d2de6c0f-4a95-4100-a495-47e47c7d1575\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x" Jul 23 18:06:23.013496 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:23.013462 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fmdxr" event={"ID":"5fe90316-7b4f-42cb-aeee-1e3e53ae8012","Type":"ContainerStarted","Data":"3874fbb745287d11cbb29581e191810afac6dd7a169def379d21225dcd39e3f4"} Jul 23 18:06:23.013664 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:23.013505 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fmdxr" event={"ID":"5fe90316-7b4f-42cb-aeee-1e3e53ae8012","Type":"ContainerStarted","Data":"bcd3bb13f309f64d049d599638cf32df2a6e16c4e42a2df5f44c4de910efba04"} Jul 23 18:06:23.034895 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:23.034844 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-fmdxr" podStartSLOduration=2.034827643 podStartE2EDuration="2.034827643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:06:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:23.032274171 +0000 UTC m=+404.519766163" watchObservedRunningTime="2026-07-23 18:06:23.034827643 +0000 UTC m=+404.522319634" Jul 23 18:06:23.036775 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:23.036753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2de6c0f-4a95-4100-a495-47e47c7d1575-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7tt9x\" (UID: \"d2de6c0f-4a95-4100-a495-47e47c7d1575\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x" Jul 23 18:06:23.036835 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:23.036788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfjz2\" (UniqueName: \"kubernetes.io/projected/d2de6c0f-4a95-4100-a495-47e47c7d1575-kube-api-access-kfjz2\") pod \"openshift-lws-operator-fd8ccff4c-7tt9x\" (UID: \"d2de6c0f-4a95-4100-a495-47e47c7d1575\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x" Jul 23 18:06:23.037137 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:23.037119 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2de6c0f-4a95-4100-a495-47e47c7d1575-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7tt9x\" (UID: \"d2de6c0f-4a95-4100-a495-47e47c7d1575\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x" Jul 23 18:06:23.049991 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:23.049961 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfjz2\" (UniqueName: \"kubernetes.io/projected/d2de6c0f-4a95-4100-a495-47e47c7d1575-kube-api-access-kfjz2\") pod \"openshift-lws-operator-fd8ccff4c-7tt9x\" (UID: \"d2de6c0f-4a95-4100-a495-47e47c7d1575\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x" Jul 23 18:06:23.216318 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:23.216286 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x" Jul 23 18:06:23.348709 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:23.348683 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x"] Jul 23 18:06:23.350854 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:06:23.350827 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2de6c0f_4a95_4100_a495_47e47c7d1575.slice/crio-ee6d48cb49a951019cdbb6e784f75f63d615bf15a8b2d6da585120570fb525e8 WatchSource:0}: Error finding container ee6d48cb49a951019cdbb6e784f75f63d615bf15a8b2d6da585120570fb525e8: Status 404 returned error can't find the container with id ee6d48cb49a951019cdbb6e784f75f63d615bf15a8b2d6da585120570fb525e8 Jul 23 18:06:24.021161 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:24.021054 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x" event={"ID":"d2de6c0f-4a95-4100-a495-47e47c7d1575","Type":"ContainerStarted","Data":"ee6d48cb49a951019cdbb6e784f75f63d615bf15a8b2d6da585120570fb525e8"} Jul 23 18:06:26.032053 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:26.031951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x" event={"ID":"d2de6c0f-4a95-4100-a495-47e47c7d1575","Type":"ContainerStarted","Data":"606fcd42f2c9ae863a73b84cbce9c0a812e179b3edd57f02321742095e90c058"} Jul 23 18:06:26.054486 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:26.054433 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7tt9x" podStartSLOduration=1.6417031450000001 podStartE2EDuration="4.05441479s" totalImagesPullingTime="2.412711645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:26.052537163 +0000 UTC m=+407.540029156" watchObservedRunningTime="2026-07-23 18:06:26.05441479 +0000 UTC m=+407.541906783" Jul 23 18:06:26.888935 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:26.888897 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:27.093645 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:27.093609 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:29.015751 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:29.015685 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:44.520747 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:44.520704 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:44.617899 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:44.617862 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:45.060940 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:45.060903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:45.165577 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:45.165543 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:45.718533 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:45.718498 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:46.044790 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.044701 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:46.153509 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.153475 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:46.236436 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.236402 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt"] Jul 23 18:06:46.243522 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.243505 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.247653 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.247631 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 18:06:46.247878 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.247857 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g6dq5\"" Jul 23 18:06:46.247969 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.247955 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 18:06:46.248163 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.248150 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 18:06:46.248233 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.248194 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 18:06:46.261656 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.261628 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt"] Jul 23 18:06:46.324590 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.324503 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d2e60fe4-ac96-4576-8cba-da52a5e531eb-manifests\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.324733 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.324590 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2e60fe4-ac96-4576-8cba-da52a5e531eb-tmp\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.324733 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.324616 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2e60fe4-ac96-4576-8cba-da52a5e531eb-webhook-cert\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.324733 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.324672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d2e60fe4-ac96-4576-8cba-da52a5e531eb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.324733 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.324694 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsj2l\" (UniqueName: \"kubernetes.io/projected/d2e60fe4-ac96-4576-8cba-da52a5e531eb-kube-api-access-bsj2l\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.401067 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.401027 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:46.425150 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.425121 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d2e60fe4-ac96-4576-8cba-da52a5e531eb-manifests\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.425278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.425161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2e60fe4-ac96-4576-8cba-da52a5e531eb-tmp\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.425278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.425180 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2e60fe4-ac96-4576-8cba-da52a5e531eb-webhook-cert\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.425278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.425240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d2e60fe4-ac96-4576-8cba-da52a5e531eb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.425278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.425257 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsj2l\" (UniqueName: \"kubernetes.io/projected/d2e60fe4-ac96-4576-8cba-da52a5e531eb-kube-api-access-bsj2l\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.425644 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.425610 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d2e60fe4-ac96-4576-8cba-da52a5e531eb-manifests\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.427545 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.427526 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2e60fe4-ac96-4576-8cba-da52a5e531eb-tmp\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.427674 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.427656 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2e60fe4-ac96-4576-8cba-da52a5e531eb-webhook-cert\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.427761 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.427741 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d2e60fe4-ac96-4576-8cba-da52a5e531eb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.435538 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.435516 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsj2l\" (UniqueName: \"kubernetes.io/projected/d2e60fe4-ac96-4576-8cba-da52a5e531eb-kube-api-access-bsj2l\") pod \"opendatahub-operator-controller-manager-657dbb4b4c-rkhxt\" (UID: \"d2e60fe4-ac96-4576-8cba-da52a5e531eb\") " pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.549825 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.549785 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:46.554072 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.554042 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:46.655972 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.655932 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:46.695009 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.694882 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt"] Jul 23 18:06:46.697283 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:06:46.697252 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e60fe4_ac96_4576_8cba_da52a5e531eb.slice/crio-80eb880974ca5a0fa770823bc677537742c04c7e1c8e8bd40b26e7b3887e3bec WatchSource:0}: Error finding container 80eb880974ca5a0fa770823bc677537742c04c7e1c8e8bd40b26e7b3887e3bec: Status 404 returned error can't find the container with id 80eb880974ca5a0fa770823bc677537742c04c7e1c8e8bd40b26e7b3887e3bec Jul 23 18:06:46.755765 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.755729 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:46.854240 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.854136 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:46.957451 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:46.957418 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:47.121894 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:47.121815 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" event={"ID":"d2e60fe4-ac96-4576-8cba-da52a5e531eb","Type":"ContainerStarted","Data":"80eb880974ca5a0fa770823bc677537742c04c7e1c8e8bd40b26e7b3887e3bec"} Jul 23 18:06:50.138232 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:50.138178 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" event={"ID":"d2e60fe4-ac96-4576-8cba-da52a5e531eb","Type":"ContainerStarted","Data":"b04e15e9df30bb29df044e14c1924f682ab4c5c2afc1ea752df59693eee1b52c"} Jul 23 18:06:51.144342 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:51.144251 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2e60fe4-ac96-4576-8cba-da52a5e531eb" containerID="b04e15e9df30bb29df044e14c1924f682ab4c5c2afc1ea752df59693eee1b52c" exitCode=0 Jul 23 18:06:51.144738 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:51.144341 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" event={"ID":"d2e60fe4-ac96-4576-8cba-da52a5e531eb","Type":"ContainerDied","Data":"b04e15e9df30bb29df044e14c1924f682ab4c5c2afc1ea752df59693eee1b52c"} Jul 23 18:06:52.150853 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:52.150823 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" event={"ID":"d2e60fe4-ac96-4576-8cba-da52a5e531eb","Type":"ContainerStarted","Data":"8aabcdb75a7d60fbbc0a8015ca70ffb1760bdb5bdcdb631927d9c13adaddb1c3"} Jul 23 18:06:52.151415 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:52.150951 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:06:52.176636 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:52.176584 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" podStartSLOduration=1.644916844 podStartE2EDuration="6.176564826s" totalImagesPullingTime="3.504638144s" totalInitContainerRuntime="1.027009838s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:06:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:52.175076548 +0000 UTC m=+433.662568541" watchObservedRunningTime="2026-07-23 18:06:52.176564826 +0000 UTC m=+433.664056817" Jul 23 18:06:55.338237 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:55.338185 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:55.502603 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:55.502571 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:57.752465 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:57.752429 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:06:57.847691 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:06:57.847659 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:03.158635 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:03.158560 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-657dbb4b4c-rkhxt" Jul 23 18:07:06.079619 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:06.079580 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:06.298230 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:06.298173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:06.878534 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:06.878499 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:06.918555 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:06.918526 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv"] Jul 23 18:07:06.921286 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:06.921266 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" Jul 23 18:07:06.926948 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:06.926916 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 18:07:06.927090 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:06.927000 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 18:07:06.927090 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:06.926915 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 18:07:06.928566 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:06.928445 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8x2lb\"" Jul 23 18:07:06.928657 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:06.928560 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 18:07:06.951953 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:06.951922 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv"] Jul 23 18:07:07.007642 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.007604 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3598fbbd-f46f-4e8e-9782-b49dd9ec1c37-tmp\") pod \"kube-auth-proxy-77d9cbf8b4-w6kwv\" (UID: \"3598fbbd-f46f-4e8e-9782-b49dd9ec1c37\") " pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" Jul 23 18:07:07.007835 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.007665 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3598fbbd-f46f-4e8e-9782-b49dd9ec1c37-tls-certs\") pod \"kube-auth-proxy-77d9cbf8b4-w6kwv\" (UID: \"3598fbbd-f46f-4e8e-9782-b49dd9ec1c37\") " pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" Jul 23 18:07:07.007835 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.007740 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6n8l\" (UniqueName: \"kubernetes.io/projected/3598fbbd-f46f-4e8e-9782-b49dd9ec1c37-kube-api-access-b6n8l\") pod \"kube-auth-proxy-77d9cbf8b4-w6kwv\" (UID: \"3598fbbd-f46f-4e8e-9782-b49dd9ec1c37\") " pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" Jul 23 18:07:07.109078 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.109035 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3598fbbd-f46f-4e8e-9782-b49dd9ec1c37-tls-certs\") pod \"kube-auth-proxy-77d9cbf8b4-w6kwv\" (UID: \"3598fbbd-f46f-4e8e-9782-b49dd9ec1c37\") " pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" Jul 23 18:07:07.109569 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.109237 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6n8l\" (UniqueName: \"kubernetes.io/projected/3598fbbd-f46f-4e8e-9782-b49dd9ec1c37-kube-api-access-b6n8l\") pod \"kube-auth-proxy-77d9cbf8b4-w6kwv\" (UID: \"3598fbbd-f46f-4e8e-9782-b49dd9ec1c37\") " pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" Jul 23 18:07:07.109569 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.109291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3598fbbd-f46f-4e8e-9782-b49dd9ec1c37-tmp\") pod \"kube-auth-proxy-77d9cbf8b4-w6kwv\" (UID: \"3598fbbd-f46f-4e8e-9782-b49dd9ec1c37\") " pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" Jul 23 18:07:07.111617 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.111594 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3598fbbd-f46f-4e8e-9782-b49dd9ec1c37-tmp\") pod \"kube-auth-proxy-77d9cbf8b4-w6kwv\" (UID: \"3598fbbd-f46f-4e8e-9782-b49dd9ec1c37\") " pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" Jul 23 18:07:07.112093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.112072 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3598fbbd-f46f-4e8e-9782-b49dd9ec1c37-tls-certs\") pod \"kube-auth-proxy-77d9cbf8b4-w6kwv\" (UID: \"3598fbbd-f46f-4e8e-9782-b49dd9ec1c37\") " pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" Jul 23 18:07:07.116412 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.116390 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:07.121591 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.121565 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6n8l\" (UniqueName: \"kubernetes.io/projected/3598fbbd-f46f-4e8e-9782-b49dd9ec1c37-kube-api-access-b6n8l\") pod \"kube-auth-proxy-77d9cbf8b4-w6kwv\" (UID: \"3598fbbd-f46f-4e8e-9782-b49dd9ec1c37\") " pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" Jul 23 18:07:07.231313 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.231272 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" Jul 23 18:07:07.276731 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.276689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:07.424933 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.424894 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv"] Jul 23 18:07:07.428006 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:07:07.427969 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3598fbbd_f46f_4e8e_9782_b49dd9ec1c37.slice/crio-e60a99dbb6a0dbd4aba14ef8d2d3c0bad4f1894291075d4ec849402eaa725bd9 WatchSource:0}: Error finding container e60a99dbb6a0dbd4aba14ef8d2d3c0bad4f1894291075d4ec849402eaa725bd9: Status 404 returned error can't find the container with id e60a99dbb6a0dbd4aba14ef8d2d3c0bad4f1894291075d4ec849402eaa725bd9 Jul 23 18:07:07.457467 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.457434 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:07.688392 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:07.688294 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:08.222309 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:08.222263 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" event={"ID":"3598fbbd-f46f-4e8e-9782-b49dd9ec1c37","Type":"ContainerStarted","Data":"e60a99dbb6a0dbd4aba14ef8d2d3c0bad4f1894291075d4ec849402eaa725bd9"} Jul 23 18:07:08.651561 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:08.651466 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:08.767528 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:08.767495 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:09.190777 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:09.190734 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:09.293026 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:09.292985 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:11.112057 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:11.112012 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:11.240479 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:11.240438 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" event={"ID":"3598fbbd-f46f-4e8e-9782-b49dd9ec1c37","Type":"ContainerStarted","Data":"e2eaddb4d73ea5152aa209588540b40ad7fde60430933a3458f712f113e5bf5c"} Jul 23 18:07:11.263380 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:11.263322 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77d9cbf8b4-w6kwv" podStartSLOduration=2.471678498 podStartE2EDuration="5.263305728s" totalImagesPullingTime="2.79162723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:11.263121507 +0000 UTC m=+452.750613497" watchObservedRunningTime="2026-07-23 18:07:11.263305728 +0000 UTC m=+452.750797720" Jul 23 18:07:14.832492 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:14.832444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:15.026793 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.026762 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:15.768762 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.768724 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq"] Jul 23 18:07:15.771299 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.771281 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.774780 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.774743 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 23 18:07:15.774914 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.774743 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-77tr6\"" Jul 23 18:07:15.774914 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.774754 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 23 18:07:15.774914 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.774799 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 23 18:07:15.790329 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.790302 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq"] Jul 23 18:07:15.803518 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.803490 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5qsr\" (UniqueName: \"kubernetes.io/projected/63eb0400-1403-45be-8d95-ea1aee7aa740-kube-api-access-r5qsr\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.803657 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.803532 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63eb0400-1403-45be-8d95-ea1aee7aa740-cert\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.803657 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.803569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/63eb0400-1403-45be-8d95-ea1aee7aa740-metrics-cert\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.803657 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.803608 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/63eb0400-1403-45be-8d95-ea1aee7aa740-manager-config\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.904524 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.904490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63eb0400-1403-45be-8d95-ea1aee7aa740-cert\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.904936 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.904540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/63eb0400-1403-45be-8d95-ea1aee7aa740-metrics-cert\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.904936 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.904583 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/63eb0400-1403-45be-8d95-ea1aee7aa740-manager-config\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.904936 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.904632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5qsr\" (UniqueName: \"kubernetes.io/projected/63eb0400-1403-45be-8d95-ea1aee7aa740-kube-api-access-r5qsr\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.905564 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.905534 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/63eb0400-1403-45be-8d95-ea1aee7aa740-manager-config\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.907240 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.907183 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/63eb0400-1403-45be-8d95-ea1aee7aa740-metrics-cert\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.907413 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.907391 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63eb0400-1403-45be-8d95-ea1aee7aa740-cert\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:15.915841 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:15.915814 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5qsr\" (UniqueName: \"kubernetes.io/projected/63eb0400-1403-45be-8d95-ea1aee7aa740-kube-api-access-r5qsr\") pod \"lws-controller-manager-7ff5b44f4d-nrgnq\" (UID: \"63eb0400-1403-45be-8d95-ea1aee7aa740\") " pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:16.025049 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:16.024954 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:16.080840 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:16.080808 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:16.246394 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:16.246169 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq"] Jul 23 18:07:16.249048 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:07:16.249023 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63eb0400_1403_45be_8d95_ea1aee7aa740.slice/crio-20ec57ccb12c4f14cf0a4b42d060fd22ff013380b1a5858f59f99bb647364fe0 WatchSource:0}: Error finding container 20ec57ccb12c4f14cf0a4b42d060fd22ff013380b1a5858f59f99bb647364fe0: Status 404 returned error can't find the container with id 20ec57ccb12c4f14cf0a4b42d060fd22ff013380b1a5858f59f99bb647364fe0 Jul 23 18:07:16.263173 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:16.263144 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" event={"ID":"63eb0400-1403-45be-8d95-ea1aee7aa740","Type":"ContainerStarted","Data":"20ec57ccb12c4f14cf0a4b42d060fd22ff013380b1a5858f59f99bb647364fe0"} Jul 23 18:07:16.318112 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:16.318001 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:16.527173 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:16.527131 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:18.275544 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:18.275508 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" event={"ID":"63eb0400-1403-45be-8d95-ea1aee7aa740","Type":"ContainerStarted","Data":"195f353a473c8d302b55e7b229a970a00ec18b280bb2ad187dafb60b0a04a368"} Jul 23 18:07:18.275906 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:18.275624 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:18.378171 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:18.378115 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" podStartSLOduration=1.7928303840000002 podStartE2EDuration="3.378098191s" totalImagesPullingTime="1.585267807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:18.376852112 +0000 UTC m=+459.864344104" watchObservedRunningTime="2026-07-23 18:07:18.378098191 +0000 UTC m=+459.865590183" Jul 23 18:07:21.156184 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:21.156143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:21.397110 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:21.397077 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:21.751286 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:21.751256 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:22.123288 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:22.123180 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:22.275311 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:22.275272 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:22.733753 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:22.733719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:29.282033 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:29.282004 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7ff5b44f4d-nrgnq" Jul 23 18:07:32.388041 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:32.388002 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:32.490789 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:32.490756 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:43.759409 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:43.759375 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:43.836583 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:07:43.836549 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:07.709360 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:07.709326 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:07.867958 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:07.867926 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:08.015081 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:08.014991 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:08.590920 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:08.590886 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:08.801246 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:08.801190 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:20.539373 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:20.539339 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:20.640958 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:20.640915 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:26.644932 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:26.644893 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:29.828598 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:29.828559 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:33.024475 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:33.024391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:35.447353 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:35.447315 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:37.134186 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:37.134157 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:37.385510 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:37.385420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:37.948372 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:37.948343 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:40.262192 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:40.262149 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:40.433378 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:40.433341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:41.163647 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:41.163605 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:43.023071 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:43.023041 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:44.113251 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:44.113194 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:44.298070 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:44.298037 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:45.120075 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:45.120042 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:45.219127 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:45.219093 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:54.021135 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:54.021099 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:54.127748 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:08:54.127708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:00.140475 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:00.140443 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:00.303586 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:00.303550 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:00.761234 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:00.761181 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:01.088114 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:01.088032 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:05.355196 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:05.355159 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:05.459919 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:05.459884 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:09.613433 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:09.613400 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:09.715699 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:09.715667 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:17.368351 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:17.368313 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:17.454812 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:17.454779 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:22.807206 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:22.807167 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:22.903060 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:22.903021 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:25.837811 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:25.837771 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:25.936287 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:25.936246 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:29.057226 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:29.057184 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:35.178519 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:35.178483 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:35.301005 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:35.300967 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:37.030204 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.030149 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:37.148563 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.148531 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:37.218973 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.218927 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:09:37.222023 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.221997 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" Jul 23 18:09:37.225394 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.225371 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 18:09:37.225510 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.225382 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 18:09:37.226259 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.226243 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 18:09:37.226317 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.226245 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lrl9q\"" Jul 23 18:09:37.235085 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.235065 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:09:37.243947 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.243923 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:37.275738 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.275710 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:09:37.344327 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.344235 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bns24\" (UniqueName: \"kubernetes.io/projected/e16edf74-ea69-4860-9043-b94ee9b4da16-kube-api-access-bns24\") pod \"limitador-limitador-54d6d988d5-q58gk\" (UID: \"e16edf74-ea69-4860-9043-b94ee9b4da16\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" Jul 23 18:09:37.344327 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.344304 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e16edf74-ea69-4860-9043-b94ee9b4da16-config-file\") pod \"limitador-limitador-54d6d988d5-q58gk\" (UID: \"e16edf74-ea69-4860-9043-b94ee9b4da16\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" Jul 23 18:09:37.444779 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.444745 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bns24\" (UniqueName: \"kubernetes.io/projected/e16edf74-ea69-4860-9043-b94ee9b4da16-kube-api-access-bns24\") pod \"limitador-limitador-54d6d988d5-q58gk\" (UID: \"e16edf74-ea69-4860-9043-b94ee9b4da16\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" Jul 23 18:09:37.444955 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.444804 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e16edf74-ea69-4860-9043-b94ee9b4da16-config-file\") pod \"limitador-limitador-54d6d988d5-q58gk\" (UID: \"e16edf74-ea69-4860-9043-b94ee9b4da16\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" Jul 23 18:09:37.445387 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.445370 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e16edf74-ea69-4860-9043-b94ee9b4da16-config-file\") pod \"limitador-limitador-54d6d988d5-q58gk\" (UID: \"e16edf74-ea69-4860-9043-b94ee9b4da16\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" Jul 23 18:09:37.456522 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.456488 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bns24\" (UniqueName: \"kubernetes.io/projected/e16edf74-ea69-4860-9043-b94ee9b4da16-kube-api-access-bns24\") pod \"limitador-limitador-54d6d988d5-q58gk\" (UID: \"e16edf74-ea69-4860-9043-b94ee9b4da16\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" Jul 23 18:09:37.532156 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.532120 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" Jul 23 18:09:37.667701 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.667655 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:09:37.669982 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:09:37.669952 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16edf74_ea69_4860_9043_b94ee9b4da16.slice/crio-b8f17e36e97d68e580ce73774756f098ccd45e95270ee74116d3968b8a88b759 WatchSource:0}: Error finding container b8f17e36e97d68e580ce73774756f098ccd45e95270ee74116d3968b8a88b759: Status 404 returned error can't find the container with id b8f17e36e97d68e580ce73774756f098ccd45e95270ee74116d3968b8a88b759 Jul 23 18:09:37.892961 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.892878 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" event={"ID":"e16edf74-ea69-4860-9043-b94ee9b4da16","Type":"ContainerStarted","Data":"b8f17e36e97d68e580ce73774756f098ccd45e95270ee74116d3968b8a88b759"} Jul 23 18:09:37.908805 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:37.908773 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:38.004063 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.004032 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:38.077987 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.077949 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-qv46c"] Jul 23 18:09:38.082562 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.082540 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qv46c" Jul 23 18:09:38.085155 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.085127 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pnkkw\"" Jul 23 18:09:38.092464 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.092441 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qv46c"] Jul 23 18:09:38.251535 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.251501 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkd9k\" (UniqueName: \"kubernetes.io/projected/bf151f8f-9cbc-470f-b02a-699cc83a6252-kube-api-access-rkd9k\") pod \"authorino-75d556bd77-qv46c\" (UID: \"bf151f8f-9cbc-470f-b02a-699cc83a6252\") " pod="kuadrant-system/authorino-75d556bd77-qv46c" Jul 23 18:09:38.352652 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.352575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkd9k\" (UniqueName: \"kubernetes.io/projected/bf151f8f-9cbc-470f-b02a-699cc83a6252-kube-api-access-rkd9k\") pod \"authorino-75d556bd77-qv46c\" (UID: \"bf151f8f-9cbc-470f-b02a-699cc83a6252\") " pod="kuadrant-system/authorino-75d556bd77-qv46c" Jul 23 18:09:38.362425 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.362378 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkd9k\" (UniqueName: \"kubernetes.io/projected/bf151f8f-9cbc-470f-b02a-699cc83a6252-kube-api-access-rkd9k\") pod \"authorino-75d556bd77-qv46c\" (UID: \"bf151f8f-9cbc-470f-b02a-699cc83a6252\") " pod="kuadrant-system/authorino-75d556bd77-qv46c" Jul 23 18:09:38.392255 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.392203 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qv46c" Jul 23 18:09:38.574759 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.574686 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qv46c"] Jul 23 18:09:38.581703 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:09:38.581672 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf151f8f_9cbc_470f_b02a_699cc83a6252.slice/crio-840c6f31b340206ab8b6c156c4ef48171bb04e32bbc8b32ed28f3faaa3f1f458 WatchSource:0}: Error finding container 840c6f31b340206ab8b6c156c4ef48171bb04e32bbc8b32ed28f3faaa3f1f458: Status 404 returned error can't find the container with id 840c6f31b340206ab8b6c156c4ef48171bb04e32bbc8b32ed28f3faaa3f1f458 Jul 23 18:09:38.900482 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.900392 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qv46c" event={"ID":"bf151f8f-9cbc-470f-b02a-699cc83a6252","Type":"ContainerStarted","Data":"840c6f31b340206ab8b6c156c4ef48171bb04e32bbc8b32ed28f3faaa3f1f458"} Jul 23 18:09:38.954305 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.954183 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:09:38.958051 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:38.958027 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:09:39.103370 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:39.103334 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:39.176144 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:39.176011 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:42.925792 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:42.925754 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" event={"ID":"e16edf74-ea69-4860-9043-b94ee9b4da16","Type":"ContainerStarted","Data":"20395ac3fbca2305e97502478cbb7a4d3092b13cf4cb89f65ab897bb8391f5f8"} Jul 23 18:09:42.926296 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:42.925826 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" Jul 23 18:09:42.927223 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:42.927179 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qv46c" event={"ID":"bf151f8f-9cbc-470f-b02a-699cc83a6252","Type":"ContainerStarted","Data":"41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7"} Jul 23 18:09:42.945813 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:42.945772 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" podStartSLOduration=1.367783346 podStartE2EDuration="5.945758358s" totalImagesPullingTime="4.577975012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:09:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:09:42.944008102 +0000 UTC m=+604.431500097" watchObservedRunningTime="2026-07-23 18:09:42.945758358 +0000 UTC m=+604.433250400" Jul 23 18:09:42.967908 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:42.967862 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-qv46c" podStartSLOduration=1.300583956 podStartE2EDuration="4.967847752s" totalImagesPullingTime="3.667263796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:09:42.966810495 +0000 UTC m=+604.454302488" watchObservedRunningTime="2026-07-23 18:09:42.967847752 +0000 UTC m=+604.455339745" Jul 23 18:09:53.935935 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:53.935904 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-q58gk" Jul 23 18:09:58.347234 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:58.347187 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:58.455380 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:58.455347 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:58.842353 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:09:58.842292 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:10.856615 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:10.856519 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:11.043293 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:11.043256 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:12.578957 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:12.578910 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qv46c"] Jul 23 18:10:12.579490 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:12.579090 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-qv46c" podUID="bf151f8f-9cbc-470f-b02a-699cc83a6252" containerName="authorino" containerID="cri-o://41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7" gracePeriod=30 Jul 23 18:10:12.827896 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:12.827868 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qv46c" Jul 23 18:10:12.965862 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:12.965834 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkd9k\" (UniqueName: \"kubernetes.io/projected/bf151f8f-9cbc-470f-b02a-699cc83a6252-kube-api-access-rkd9k\") pod \"bf151f8f-9cbc-470f-b02a-699cc83a6252\" (UID: \"bf151f8f-9cbc-470f-b02a-699cc83a6252\") " Jul 23 18:10:12.968010 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:12.967976 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf151f8f-9cbc-470f-b02a-699cc83a6252-kube-api-access-rkd9k" (OuterVolumeSpecName: "kube-api-access-rkd9k") pod "bf151f8f-9cbc-470f-b02a-699cc83a6252" (UID: "bf151f8f-9cbc-470f-b02a-699cc83a6252"). InnerVolumeSpecName "kube-api-access-rkd9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:10:13.066964 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:13.066928 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkd9k\" (UniqueName: \"kubernetes.io/projected/bf151f8f-9cbc-470f-b02a-699cc83a6252-kube-api-access-rkd9k\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:10:13.068866 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:13.068841 2562 generic.go:356] "Generic (PLEG): container finished" podID="bf151f8f-9cbc-470f-b02a-699cc83a6252" containerID="41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7" exitCode=0 Jul 23 18:10:13.069002 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:13.068886 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qv46c" Jul 23 18:10:13.069002 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:13.068931 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qv46c" event={"ID":"bf151f8f-9cbc-470f-b02a-699cc83a6252","Type":"ContainerDied","Data":"41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7"} Jul 23 18:10:13.069002 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:13.068970 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qv46c" event={"ID":"bf151f8f-9cbc-470f-b02a-699cc83a6252","Type":"ContainerDied","Data":"840c6f31b340206ab8b6c156c4ef48171bb04e32bbc8b32ed28f3faaa3f1f458"} Jul 23 18:10:13.069117 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:13.068987 2562 scope.go:117] "RemoveContainer" containerID="41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7" Jul 23 18:10:13.077751 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:13.077736 2562 scope.go:117] "RemoveContainer" containerID="41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7" Jul 23 18:10:13.078007 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:10:13.077986 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7\": container with ID starting with 41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7 not found: ID does not exist" containerID="41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7" Jul 23 18:10:13.078055 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:13.078014 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7"} err="failed to get container status \"41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7\": rpc error: code = NotFound desc = could not find container \"41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7\": container with ID starting with 41a278d907ba1b21bb32bc810dc1568804535c83e155726dcf744e8a5aef79d7 not found: ID does not exist" Jul 23 18:10:13.090527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:13.090502 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qv46c"] Jul 23 18:10:13.094150 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:13.094129 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qv46c"] Jul 23 18:10:13.136639 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:13.136608 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf151f8f-9cbc-470f-b02a-699cc83a6252" path="/var/lib/kubelet/pods/bf151f8f-9cbc-470f-b02a-699cc83a6252/volumes" Jul 23 18:10:17.500824 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:17.500786 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:17.617334 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:17.617271 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:17.721606 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:17.721575 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:17.816820 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:17.816736 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:20.634357 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:20.634321 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:20.789253 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:20.789198 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:29.549553 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:29.549512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:50.605876 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:50.605828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:50.721972 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:50.721940 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:51.312400 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:51.312367 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:51.670012 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:51.669928 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:52.115821 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:52.115783 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:52.249782 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:52.249748 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:52.338716 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:10:52.338671 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:11:01.415278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:01.415208 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:03.011931 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:03.011897 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:15.607836 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:15.607801 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:16.109622 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:16.109588 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:11:28.307616 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:28.307576 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:28.764048 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:28.764006 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:11:38.819612 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:38.819515 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:39.894740 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:39.894705 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:11:46.002400 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:46.002365 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:47.458690 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:47.458645 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:11:58.156856 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.156819 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g"] Jul 23 18:11:58.157205 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.157158 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf151f8f-9cbc-470f-b02a-699cc83a6252" containerName="authorino" Jul 23 18:11:58.157205 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.157169 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf151f8f-9cbc-470f-b02a-699cc83a6252" containerName="authorino" Jul 23 18:11:58.157308 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.157246 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf151f8f-9cbc-470f-b02a-699cc83a6252" containerName="authorino" Jul 23 18:11:58.159458 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.159441 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.162000 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.161978 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qc4rc\"" Jul 23 18:11:58.162305 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.162286 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 18:11:58.162885 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.162869 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 18:11:58.163092 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.163074 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 23 18:11:58.176097 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.176063 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.176097 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.176104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.176276 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.176143 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22778797-4630-465c-896d-5b7e42b44bef-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.176276 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.176183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.176276 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.176201 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.176276 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.176254 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d5zh\" (UniqueName: \"kubernetes.io/projected/22778797-4630-465c-896d-5b7e42b44bef-kube-api-access-5d5zh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.195205 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.195180 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g"] Jul 23 18:11:58.218899 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.218870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:58.276695 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.276668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.276826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.276700 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.276826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.276747 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22778797-4630-465c-896d-5b7e42b44bef-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.276826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.276776 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.276826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.276796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.276826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.276822 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d5zh\" (UniqueName: \"kubernetes.io/projected/22778797-4630-465c-896d-5b7e42b44bef-kube-api-access-5d5zh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.277090 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.277066 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.277194 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.277178 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.277274 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.277242 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.279018 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.278999 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22778797-4630-465c-896d-5b7e42b44bef-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.279298 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.279279 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22778797-4630-465c-896d-5b7e42b44bef-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.287309 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.287287 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d5zh\" (UniqueName: \"kubernetes.io/projected/22778797-4630-465c-896d-5b7e42b44bef-kube-api-access-5d5zh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2x6g\" (UID: \"22778797-4630-465c-896d-5b7e42b44bef\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.469714 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.469682 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:11:58.626440 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.626414 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g"] Jul 23 18:11:58.627751 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:11:58.627725 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22778797_4630_465c_896d_5b7e42b44bef.slice/crio-d103e8896be792635aeae62e2a952c7888a3115deee70fa684edba250df4c1bb WatchSource:0}: Error finding container d103e8896be792635aeae62e2a952c7888a3115deee70fa684edba250df4c1bb: Status 404 returned error can't find the container with id d103e8896be792635aeae62e2a952c7888a3115deee70fa684edba250df4c1bb Jul 23 18:11:58.629129 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:58.629113 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 18:11:59.551711 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:11:59.551670 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" event={"ID":"22778797-4630-465c-896d-5b7e42b44bef","Type":"ContainerStarted","Data":"d103e8896be792635aeae62e2a952c7888a3115deee70fa684edba250df4c1bb"} Jul 23 18:12:00.557768 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:00.557734 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" event={"ID":"22778797-4630-465c-896d-5b7e42b44bef","Type":"ContainerStarted","Data":"aa9d24090ea0712b1078de058b6eb3899e30920c51980bbabe30c0b5ef407a1c"} Jul 23 18:12:00.577773 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:00.577730 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" podStartSLOduration=1.6219336229999999 podStartE2EDuration="2.57771377s" totalImagesPullingTime="955.780147ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:12:00.576045181 +0000 UTC m=+742.063537171" watchObservedRunningTime="2026-07-23 18:12:00.57771377 +0000 UTC m=+742.065205791" Jul 23 18:12:00.970297 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:00.970253 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:12:05.263141 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:05.263107 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:05.563258 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:05.563161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:07.445198 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.445167 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj"] Jul 23 18:12:07.448021 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.448001 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.450824 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.450807 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 23 18:12:07.466920 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.466895 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj"] Jul 23 18:12:07.477297 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.477268 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:07.554304 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.554269 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.554466 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.554310 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.554466 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.554327 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.554466 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.554401 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.554466 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.554430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be16716d-7764-43ba-aed9-3c795eff521c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.554595 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.554489 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn6w4\" (UniqueName: \"kubernetes.io/projected/be16716d-7764-43ba-aed9-3c795eff521c-kube-api-access-bn6w4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.655504 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.655464 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn6w4\" (UniqueName: \"kubernetes.io/projected/be16716d-7764-43ba-aed9-3c795eff521c-kube-api-access-bn6w4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.655686 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.655547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.655686 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.655580 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.655686 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.655602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.655817 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.655726 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.655817 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.655757 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be16716d-7764-43ba-aed9-3c795eff521c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.656016 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.655994 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.656093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.656043 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.656093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.656075 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.657943 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.657916 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be16716d-7764-43ba-aed9-3c795eff521c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.658250 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.658230 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be16716d-7764-43ba-aed9-3c795eff521c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.678129 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.678094 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn6w4\" (UniqueName: \"kubernetes.io/projected/be16716d-7764-43ba-aed9-3c795eff521c-kube-api-access-bn6w4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj\" (UID: \"be16716d-7764-43ba-aed9-3c795eff521c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.756759 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.756723 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:07.892962 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:07.892918 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj"] Jul 23 18:12:07.895707 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:12:07.895678 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe16716d_7764_43ba_aed9_3c795eff521c.slice/crio-e920a237c89d02926f509ee76967b28cd9a6e902ee5146bce2ab48522509bec5 WatchSource:0}: Error finding container e920a237c89d02926f509ee76967b28cd9a6e902ee5146bce2ab48522509bec5: Status 404 returned error can't find the container with id e920a237c89d02926f509ee76967b28cd9a6e902ee5146bce2ab48522509bec5 Jul 23 18:12:08.470000 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:08.469972 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:12:08.470394 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:08.470053 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:12:08.482188 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:08.482163 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:12:08.597912 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:08.597876 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" event={"ID":"be16716d-7764-43ba-aed9-3c795eff521c","Type":"ContainerStarted","Data":"f56c1a08af2e98b51e0ec07624a9df5ce90d40cbb1ae5cade3a015539ffc71f7"} Jul 23 18:12:08.598088 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:08.597922 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" event={"ID":"be16716d-7764-43ba-aed9-3c795eff521c","Type":"ContainerStarted","Data":"e920a237c89d02926f509ee76967b28cd9a6e902ee5146bce2ab48522509bec5"} Jul 23 18:12:08.608788 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:08.608766 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2x6g" Jul 23 18:12:08.624084 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:08.624034 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" podStartSLOduration=1.443220985 podStartE2EDuration="1.624017227s" totalImagesPullingTime="180.796242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:12:08.621291299 +0000 UTC m=+750.108783290" watchObservedRunningTime="2026-07-23 18:12:08.624017227 +0000 UTC m=+750.111509217" Jul 23 18:12:10.582401 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:10.582365 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:12:17.757204 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:17.757163 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:17.757605 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:17.757243 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:17.769272 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:17.769239 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:12:18.657467 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:12:18.657434 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj" Jul 23 18:13:05.484623 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.484529 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6ddffd9688-gm2xx"] Jul 23 18:13:05.488089 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.488070 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6ddffd9688-gm2xx"] Jul 23 18:13:05.488165 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.488155 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddffd9688-gm2xx" Jul 23 18:13:05.490918 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.490893 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pnkkw\"" Jul 23 18:13:05.491039 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.490893 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 23 18:13:05.554713 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.554679 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6-tls-cert\") pod \"authorino-6ddffd9688-gm2xx\" (UID: \"5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6\") " pod="kuadrant-system/authorino-6ddffd9688-gm2xx" Jul 23 18:13:05.554713 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.554715 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfcnh\" (UniqueName: \"kubernetes.io/projected/5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6-kube-api-access-nfcnh\") pod \"authorino-6ddffd9688-gm2xx\" (UID: \"5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6\") " pod="kuadrant-system/authorino-6ddffd9688-gm2xx" Jul 23 18:13:05.655532 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.655488 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6-tls-cert\") pod \"authorino-6ddffd9688-gm2xx\" (UID: \"5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6\") " pod="kuadrant-system/authorino-6ddffd9688-gm2xx" Jul 23 18:13:05.655532 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.655528 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfcnh\" (UniqueName: \"kubernetes.io/projected/5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6-kube-api-access-nfcnh\") pod \"authorino-6ddffd9688-gm2xx\" (UID: \"5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6\") " pod="kuadrant-system/authorino-6ddffd9688-gm2xx" Jul 23 18:13:05.657879 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.657850 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6-tls-cert\") pod \"authorino-6ddffd9688-gm2xx\" (UID: \"5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6\") " pod="kuadrant-system/authorino-6ddffd9688-gm2xx" Jul 23 18:13:05.666627 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.666605 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfcnh\" (UniqueName: \"kubernetes.io/projected/5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6-kube-api-access-nfcnh\") pod \"authorino-6ddffd9688-gm2xx\" (UID: \"5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6\") " pod="kuadrant-system/authorino-6ddffd9688-gm2xx" Jul 23 18:13:05.798396 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.798314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddffd9688-gm2xx" Jul 23 18:13:05.859624 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.859551 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:05.937722 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:05.937689 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6ddffd9688-gm2xx"] Jul 23 18:13:05.938869 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:13:05.938842 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f8a3f95_8ba6_4c78_aad5_3b3bbaaf24c6.slice/crio-abe3311f7911b3324a9acae0efe7ed1779f4ba05c022832330547f78995b851b WatchSource:0}: Error finding container abe3311f7911b3324a9acae0efe7ed1779f4ba05c022832330547f78995b851b: Status 404 returned error can't find the container with id abe3311f7911b3324a9acae0efe7ed1779f4ba05c022832330547f78995b851b Jul 23 18:13:06.006063 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:06.006034 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:06.866932 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:06.866888 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddffd9688-gm2xx" event={"ID":"5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6","Type":"ContainerStarted","Data":"64ef9ca28a2c4099defe9106cbd59e11ba5bc6d9d60449a9a021db51d57738c7"} Jul 23 18:13:06.867363 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:06.866943 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddffd9688-gm2xx" event={"ID":"5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6","Type":"ContainerStarted","Data":"abe3311f7911b3324a9acae0efe7ed1779f4ba05c022832330547f78995b851b"} Jul 23 18:13:06.886700 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:06.886649 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6ddffd9688-gm2xx" podStartSLOduration=1.496362395 podStartE2EDuration="1.886630968s" totalImagesPullingTime="390.268573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:13:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:13:06.885483259 +0000 UTC m=+808.372975270" watchObservedRunningTime="2026-07-23 18:13:06.886630968 +0000 UTC m=+808.374122958" Jul 23 18:13:07.057286 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:07.057249 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:07.218458 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:07.218420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:36.989973 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:36.989941 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:13:46.366671 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:46.366633 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:13:56.269996 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:56.269955 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:13:59.740889 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:59.740850 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:59.849943 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:13:59.849905 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:12.409390 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:14:12.409351 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:12.509736 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:14:12.509702 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:36.030621 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:14:36.030534 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:38.989647 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:14:38.989620 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:14:38.990129 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:14:38.990110 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:15:12.658171 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:15:12.658138 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:14.228413 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:15:14.228376 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:36.259193 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:15:36.259152 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:15:37.374737 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:15:37.374679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:47.960485 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:15:47.960445 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:15:54.459033 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:15:54.458996 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:15:56.117142 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:15:56.117102 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:56.230315 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:15:56.230281 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:56.413846 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:15:56.413727 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:02.304039 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:02.303951 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:02.478713 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:02.478677 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:02.569058 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:02.568969 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:02.678143 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:02.678105 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:05.161672 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:05.161627 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:16:08.809722 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:08.809677 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:10.169665 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:10.169622 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:10.754164 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:10.754115 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:26.158289 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:26.158247 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:16:35.553672 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:35.553629 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:16:44.261942 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:44.261893 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:16:53.451281 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:16:53.451239 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:17:04.064180 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:17:04.064137 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:17:15.963836 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:17:15.963797 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:17:26.752829 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:17:26.752782 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:17:36.247396 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:17:36.247310 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:17:58.965797 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:17:58.965759 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:18:11.459014 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:18:11.458973 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:18:15.542096 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:18:15.542057 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:18:15.628621 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:18:15.628580 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:18:34.254102 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:18:34.254055 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:18:44.652653 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:18:44.652615 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:18:51.164044 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:18:51.163987 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:19:02.756088 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:19:02.755996 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:19:14.656382 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:19:14.656344 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:19:23.965538 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:19:23.965498 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:19:32.247373 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:19:32.247331 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:19:39.019681 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:19:39.019649 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:19:39.021190 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:19:39.021167 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:19:40.863586 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:19:40.863544 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:20:47.738297 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:20:47.738259 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:30.462628 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:21:30.462583 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:21:39.452900 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:21:39.452858 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:21:42.983985 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:21:42.983943 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:52.252324 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:21:52.252283 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:22:04.158393 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:22:04.158310 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:22:14.147522 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:22:14.147482 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:22:25.147591 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:22:25.147551 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:22:33.952977 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:22:33.952933 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:22:44.459313 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:22:44.459276 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:22:56.254002 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:22:56.253956 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:23:04.152577 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:04.152531 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:23:13.049995 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:13.049950 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:23:22.347610 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:22.347561 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:23:39.357759 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:39.357661 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:23:49.263042 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:49.262998 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:23:54.862527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:54.862488 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf"] Jul 23 18:23:54.865968 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:54.865949 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:54.868575 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:54.868548 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-f9e6e192-kserve-self-signed-certs\"" Jul 23 18:23:54.887613 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:54.887514 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf"] Jul 23 18:23:54.925248 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:54.925173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:23:54.944021 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:54.943987 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-dshm\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:54.944141 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:54.944061 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-home\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:54.944141 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:54.944090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xsm9\" (UniqueName: \"kubernetes.io/projected/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-kube-api-access-5xsm9\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:54.944141 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:54.944117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tls-certs\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:54.944318 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:54.944165 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-model-cache\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:54.944318 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:54.944189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tmp-dir\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.045511 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.045472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-dshm\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.045705 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.045615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-home\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.045705 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.045637 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xsm9\" (UniqueName: \"kubernetes.io/projected/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-kube-api-access-5xsm9\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.045705 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.045661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tls-certs\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.045705 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.045687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-model-cache\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.045953 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.045707 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tmp-dir\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.046083 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.046060 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-home\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.046159 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.046115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tmp-dir\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.046203 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.046181 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-model-cache\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.047906 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.047886 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-dshm\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.048187 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.048165 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tls-certs\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.053783 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.053756 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xsm9\" (UniqueName: \"kubernetes.io/projected/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-kube-api-access-5xsm9\") pod \"e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.175866 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.175749 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:23:55.318485 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:55.318456 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf"] Jul 23 18:23:55.320650 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:23:55.320620 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e0e4b65_5002_40ae_bfaa_eeaabd74c46e.slice/crio-8fa098b353efbde1947eabab47102482fddeefe9cffe77ab9e7e68209da9d49f WatchSource:0}: Error finding container 8fa098b353efbde1947eabab47102482fddeefe9cffe77ab9e7e68209da9d49f: Status 404 returned error can't find the container with id 8fa098b353efbde1947eabab47102482fddeefe9cffe77ab9e7e68209da9d49f Jul 23 18:23:56.074145 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:56.074099 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" event={"ID":"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e","Type":"ContainerStarted","Data":"c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284"} Jul 23 18:23:56.074589 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:56.074157 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" event={"ID":"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e","Type":"ContainerStarted","Data":"8fa098b353efbde1947eabab47102482fddeefe9cffe77ab9e7e68209da9d49f"} Jul 23 18:23:56.093977 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:23:56.093921 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" podStartSLOduration=2.093903196 podStartE2EDuration="2.093903196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:23:56.092683679 +0000 UTC m=+1457.580175671" watchObservedRunningTime="2026-07-23 18:23:56.093903196 +0000 UTC m=+1457.581395187" Jul 23 18:24:05.176575 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:05.176470 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:24:05.176575 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:05.176536 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:24:05.189296 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:05.189260 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:24:05.251993 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:05.251934 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:24:06.134741 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.134709 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:24:06.254143 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.254105 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn"] Jul 23 18:24:06.258396 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.258371 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.260733 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.260707 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-f9e6e192-kserve-self-signed-certs\"" Jul 23 18:24:06.267409 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.267386 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn"] Jul 23 18:24:06.280815 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.280772 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:24:06.350400 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.350353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z765r\" (UniqueName: \"kubernetes.io/projected/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-kube-api-access-z765r\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.350582 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.350405 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-dshm\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.350582 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.350553 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-home\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.350656 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.350582 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-model-cache\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.350656 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.350606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tls-certs\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.350726 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.350675 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tmp-dir\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.452161 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.452118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-home\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.452161 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.452163 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-model-cache\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.452433 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.452193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tls-certs\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.452433 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.452265 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tmp-dir\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.452433 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.452350 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z765r\" (UniqueName: \"kubernetes.io/projected/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-kube-api-access-z765r\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.452433 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.452375 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-dshm\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.452646 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.452595 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-home\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.452704 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.452664 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-model-cache\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.452704 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.452681 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tmp-dir\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.455245 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.455185 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-dshm\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.455380 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.455364 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tls-certs\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.466544 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.466516 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z765r\" (UniqueName: \"kubernetes.io/projected/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-kube-api-access-z765r\") pod \"e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.569070 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.569026 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:06.755099 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:06.755065 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn"] Jul 23 18:24:06.758180 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:24:06.758151 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f7348d9_4915_4d8e_b00b_2a56714dc0d3.slice/crio-712064ffe3e981db732effd7d8b086af215036e26ec8894ebace8e23388570e1 WatchSource:0}: Error finding container 712064ffe3e981db732effd7d8b086af215036e26ec8894ebace8e23388570e1: Status 404 returned error can't find the container with id 712064ffe3e981db732effd7d8b086af215036e26ec8894ebace8e23388570e1 Jul 23 18:24:07.131581 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:07.131488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" event={"ID":"6f7348d9-4915-4d8e-b00b-2a56714dc0d3","Type":"ContainerStarted","Data":"b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4"} Jul 23 18:24:07.131581 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:07.131537 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" event={"ID":"6f7348d9-4915-4d8e-b00b-2a56714dc0d3","Type":"ContainerStarted","Data":"712064ffe3e981db732effd7d8b086af215036e26ec8894ebace8e23388570e1"} Jul 23 18:24:07.147975 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:07.147915 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" podStartSLOduration=1.147895733 podStartE2EDuration="1.147895733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:24:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:24:07.146587594 +0000 UTC m=+1468.634079604" watchObservedRunningTime="2026-07-23 18:24:07.147895733 +0000 UTC m=+1468.635387702" Jul 23 18:24:16.569957 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:16.569910 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:16.569957 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:16.569966 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:16.582887 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:16.582858 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:17.191755 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:17.191721 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:17.948788 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:17.948748 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:24:30.395629 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:30.395588 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf"] Jul 23 18:24:31.115721 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:31.115670 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:24:31.352629 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:31.352555 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn"] Jul 23 18:24:31.373720 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:31.373621 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:24:34.151463 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:34.151415 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:24:39.064688 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:39.064655 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:24:39.066793 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:39.066761 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:24:45.417024 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.416949 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" podUID="0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" containerName="main" containerID="cri-o://c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284" gracePeriod=45 Jul 23 18:24:45.680640 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.680565 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:24:45.726806 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.726759 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tls-certs\") pod \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " Jul 23 18:24:45.726999 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.726846 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-home\") pod \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " Jul 23 18:24:45.726999 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.726866 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-model-cache\") pod \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " Jul 23 18:24:45.726999 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.726903 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tmp-dir\") pod \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " Jul 23 18:24:45.726999 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.726924 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xsm9\" (UniqueName: \"kubernetes.io/projected/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-kube-api-access-5xsm9\") pod \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " Jul 23 18:24:45.726999 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.726965 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-dshm\") pod \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\" (UID: \"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e\") " Jul 23 18:24:45.727284 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.727114 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-model-cache" (OuterVolumeSpecName: "model-cache") pod "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" (UID: "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:24:45.727284 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.727139 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-home" (OuterVolumeSpecName: "home") pod "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" (UID: "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:24:45.727284 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.727205 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" (UID: "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:24:45.727426 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.727330 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-home\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:45.727426 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.727352 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-model-cache\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:45.727426 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.727370 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tmp-dir\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:45.729182 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.729148 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-dshm" (OuterVolumeSpecName: "dshm") pod "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" (UID: "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:24:45.729548 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.729517 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" (UID: "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:24:45.729548 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.729519 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-kube-api-access-5xsm9" (OuterVolumeSpecName: "kube-api-access-5xsm9") pod "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" (UID: "0e0e4b65-5002-40ae-bfaa-eeaabd74c46e"). InnerVolumeSpecName "kube-api-access-5xsm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:24:45.828550 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.828510 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-tls-certs\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:45.828550 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.828543 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xsm9\" (UniqueName: \"kubernetes.io/projected/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-kube-api-access-5xsm9\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:45.828550 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.828553 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e-dshm\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:45.851599 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:45.851561 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:24:46.331008 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.330974 2562 generic.go:356] "Generic (PLEG): container finished" podID="0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" containerID="c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284" exitCode=0 Jul 23 18:24:46.331182 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.331016 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" event={"ID":"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e","Type":"ContainerDied","Data":"c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284"} Jul 23 18:24:46.331182 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.331018 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" Jul 23 18:24:46.331182 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.331052 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf" event={"ID":"0e0e4b65-5002-40ae-bfaa-eeaabd74c46e","Type":"ContainerDied","Data":"8fa098b353efbde1947eabab47102482fddeefe9cffe77ab9e7e68209da9d49f"} Jul 23 18:24:46.331182 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.331077 2562 scope.go:117] "RemoveContainer" containerID="c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284" Jul 23 18:24:46.341069 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.341046 2562 scope.go:117] "RemoveContainer" containerID="c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284" Jul 23 18:24:46.341447 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:24:46.341418 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284\": container with ID starting with c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284 not found: ID does not exist" containerID="c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284" Jul 23 18:24:46.341544 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.341461 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284"} err="failed to get container status \"c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284\": rpc error: code = NotFound desc = could not find container \"c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284\": container with ID starting with c026328ec16ce2b1674f2037353aa794dc831cfbd0fdb8cc911818bb4e339284 not found: ID does not exist" Jul 23 18:24:46.352136 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.352102 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf"] Jul 23 18:24:46.355764 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.355736 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-f9e6e192-kserve-7d4d96c4c-jzjbf"] Jul 23 18:24:46.376622 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.376581 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" podUID="6f7348d9-4915-4d8e-b00b-2a56714dc0d3" containerName="main" containerID="cri-o://b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4" gracePeriod=45 Jul 23 18:24:46.636033 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.635998 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:46.736178 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.736137 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-home\") pod \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " Jul 23 18:24:46.736178 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.736182 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tmp-dir\") pod \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " Jul 23 18:24:46.736453 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.736276 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z765r\" (UniqueName: \"kubernetes.io/projected/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-kube-api-access-z765r\") pod \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " Jul 23 18:24:46.736453 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.736297 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tls-certs\") pod \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " Jul 23 18:24:46.736453 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.736315 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-model-cache\") pod \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " Jul 23 18:24:46.736453 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.736345 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-dshm\") pod \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\" (UID: \"6f7348d9-4915-4d8e-b00b-2a56714dc0d3\") " Jul 23 18:24:46.736657 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.736524 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f7348d9-4915-4d8e-b00b-2a56714dc0d3" (UID: "6f7348d9-4915-4d8e-b00b-2a56714dc0d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:24:46.736657 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.736561 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-home" (OuterVolumeSpecName: "home") pod "6f7348d9-4915-4d8e-b00b-2a56714dc0d3" (UID: "6f7348d9-4915-4d8e-b00b-2a56714dc0d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:24:46.736657 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.736634 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f7348d9-4915-4d8e-b00b-2a56714dc0d3" (UID: "6f7348d9-4915-4d8e-b00b-2a56714dc0d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:24:46.736657 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.736649 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-home\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:46.736819 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.736666 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tmp-dir\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:46.738595 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.738567 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-dshm" (OuterVolumeSpecName: "dshm") pod "6f7348d9-4915-4d8e-b00b-2a56714dc0d3" (UID: "6f7348d9-4915-4d8e-b00b-2a56714dc0d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:24:46.738708 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.738645 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f7348d9-4915-4d8e-b00b-2a56714dc0d3" (UID: "6f7348d9-4915-4d8e-b00b-2a56714dc0d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:24:46.738708 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.738688 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-kube-api-access-z765r" (OuterVolumeSpecName: "kube-api-access-z765r") pod "6f7348d9-4915-4d8e-b00b-2a56714dc0d3" (UID: "6f7348d9-4915-4d8e-b00b-2a56714dc0d3"). InnerVolumeSpecName "kube-api-access-z765r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:24:46.837518 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.837484 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z765r\" (UniqueName: \"kubernetes.io/projected/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-kube-api-access-z765r\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:46.837518 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.837511 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-tls-certs\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:46.837518 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.837521 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-model-cache\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:46.837518 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:46.837528 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7348d9-4915-4d8e-b00b-2a56714dc0d3-dshm\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:24:47.131057 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:47.131012 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" path="/var/lib/kubelet/pods/0e0e4b65-5002-40ae-bfaa-eeaabd74c46e/volumes" Jul 23 18:24:47.338071 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:47.338034 2562 generic.go:356] "Generic (PLEG): container finished" podID="6f7348d9-4915-4d8e-b00b-2a56714dc0d3" containerID="b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4" exitCode=0 Jul 23 18:24:47.338071 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:47.338072 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" Jul 23 18:24:47.338341 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:47.338122 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" event={"ID":"6f7348d9-4915-4d8e-b00b-2a56714dc0d3","Type":"ContainerDied","Data":"b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4"} Jul 23 18:24:47.338341 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:47.338167 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn" event={"ID":"6f7348d9-4915-4d8e-b00b-2a56714dc0d3","Type":"ContainerDied","Data":"712064ffe3e981db732effd7d8b086af215036e26ec8894ebace8e23388570e1"} Jul 23 18:24:47.338341 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:47.338192 2562 scope.go:117] "RemoveContainer" containerID="b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4" Jul 23 18:24:47.347477 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:47.347458 2562 scope.go:117] "RemoveContainer" containerID="b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4" Jul 23 18:24:47.347775 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:24:47.347756 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4\": container with ID starting with b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4 not found: ID does not exist" containerID="b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4" Jul 23 18:24:47.347814 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:47.347784 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4"} err="failed to get container status \"b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4\": rpc error: code = NotFound desc = could not find container \"b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4\": container with ID starting with b8c12abec0006fca326fab222d0b048b7b4c3490187832b7505a0a6cde3f0eb4 not found: ID does not exist" Jul 23 18:24:47.354152 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:47.354126 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn"] Jul 23 18:24:47.357746 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:47.357719 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-f9e6e192-kserve-7c4469f65f-kb4zn"] Jul 23 18:24:49.131321 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:49.131289 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f7348d9-4915-4d8e-b00b-2a56714dc0d3" path="/var/lib/kubelet/pods/6f7348d9-4915-4d8e-b00b-2a56714dc0d3/volumes" Jul 23 18:24:52.354864 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:24:52.354823 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:25:08.049166 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:25:08.049121 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:25:16.591557 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:25:16.591488 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:25:16.692222 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:25:16.692164 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:25:17.349169 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:25:17.349121 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:25:26.260030 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:25:26.259990 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:25:36.351787 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:25:36.351740 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:25:46.845937 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:25:46.845887 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:25:55.650439 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:25:55.650398 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:26:04.865762 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:26:04.865720 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:26:08.457927 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:26:08.457890 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:26:17.860637 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:26:17.860592 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:26:27.257443 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:26:27.257404 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:26:29.733765 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:26:29.733724 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:26:38.948866 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:26:38.948779 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:26:51.156362 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:26:51.156311 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:27:00.071084 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:27:00.071044 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:27:12.052581 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:27:12.052537 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:27:23.952935 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:27:23.952892 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:27:24.542027 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:27:24.541979 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:27:35.248584 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:27:35.248541 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:27:38.540343 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:27:38.540292 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:27:44.148654 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:27:44.148605 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:28:02.133299 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:02.133188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:05.200344 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:05.200301 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:05.304350 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:05.304310 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:05.696962 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:05.696931 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:05.765160 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:05.765114 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:28:11.898243 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:11.898194 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:11.998098 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:11.998056 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:12.171467 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:12.171377 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:12.479718 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:12.479679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:12.791286 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:12.791187 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:14.459936 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:14.459891 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:14.585474 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:14.585434 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:14.785088 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:14.784897 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:15.143634 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:15.143519 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:15.255000 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:15.254905 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:15.357714 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:15.357674 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:21.962551 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:21.962510 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:28:28.889998 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:28.889960 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:50.638143 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:50.638106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:53.856467 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:53.856431 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:28:55.169002 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:55.168961 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:55.307196 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:55.307149 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:55.558483 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:55.558447 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:55.818471 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:55.818377 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:55.915484 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:55.915446 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:56.064013 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:56.063967 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:56.375691 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:56.375654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:56.660173 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:56.660067 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:58.898163 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:58.898119 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:59.022733 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:59.022694 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:59.123367 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:59.123335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:59.520609 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:59.520560 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:28:59.620320 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:28:59.620280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:04.563424 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:04.563386 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:29:34.505391 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:34.505291 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:37.153163 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:37.153125 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:29:38.980409 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:38.980368 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:39.094470 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:39.094441 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:29:39.097531 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:39.097507 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:29:39.157550 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:39.157515 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:39.433166 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:39.433045 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:39.683916 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:39.683814 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:40.016423 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:40.016389 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:40.134965 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:40.134920 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:41.980845 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:41.980807 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:42.068092 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:42.068059 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:42.136227 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:42.136186 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:42.325805 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:42.325721 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:42.674756 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:42.674668 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:42.774761 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:42.774714 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:48.452672 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:29:48.452630 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:30:18.280650 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:18.280611 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:21.254924 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:21.254886 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:30:22.906527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:22.906482 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:24.137870 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:24.137827 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:24.372277 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:24.372204 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:24.817511 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:24.817460 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:24.927721 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:24.927681 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:25.219079 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:25.219038 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:27.174411 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:27.174377 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:27.273888 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:27.273850 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:27.360598 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:27.360553 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:27.683692 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:27.683647 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:27.983683 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:27.983641 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:28.085568 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:28.085526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:30.055306 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:30.055273 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:30:34.918070 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:34.918019 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:35.018927 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:35.018889 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:48.716503 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:48.716462 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:48.818758 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:30:48.818713 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:02.544853 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:02.544762 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:05.251550 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:05.251513 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:31:07.190822 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:07.190774 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:07.348771 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:07.348713 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:07.594717 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:07.594679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:08.228710 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:08.228670 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:08.330559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:08.330523 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:09.854838 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:09.854784 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:16.061386 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:16.061337 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:16.162867 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:16.162827 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:16.263489 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:16.263445 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:16.622818 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:16.622775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:30.340436 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:30.340396 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:42.660959 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:42.660919 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:49.164227 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:49.164165 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:49.278720 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:49.278672 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:49.366848 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:31:49.366748 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:05.365163 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:05.365122 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:11.497004 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:11.496961 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:11.601563 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:11.601530 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:11.722645 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:11.722607 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:22.300707 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:22.300667 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:23.139008 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:23.138972 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:29.615349 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:29.615304 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:30.966743 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:30.966666 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:32.708000 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:32.707964 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:34.984662 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:34.984622 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:32:38.871093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:38.871043 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:38.971315 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:38.971269 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:39.577376 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:39.577339 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:44.864646 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:44.864605 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:32:45.989104 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:45.989070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:46.094274 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:46.094242 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:46.187376 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:46.187339 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:46.571814 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:46.571777 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:46.849888 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:46.849792 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:49.621281 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:49.621246 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:49.780632 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:49.780587 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:50.073397 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:50.073360 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:50.471090 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:50.471051 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:51.121230 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:51.121168 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:51.226229 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:51.226172 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:54.133739 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:54.133701 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:54.838896 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:54.838846 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:54.927527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:54.927486 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:55.342912 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:55.342876 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:32:55.669149 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:32:55.669068 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:33:01.428279 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:01.428192 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:01.527250 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:01.527193 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:12.033612 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:12.033575 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:14.878343 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:14.878303 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:33:16.881435 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:16.881399 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:17.280808 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:17.280765 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:23.840151 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:23.840110 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:23.971989 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:23.971953 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:24.164801 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:24.164702 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:24.385967 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:24.385926 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:24.546754 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:24.546714 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:24.642477 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:24.642445 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:24.776693 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:24.776661 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:31.009207 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:31.009162 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:31.076906 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:31.076868 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:31.175020 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:31.174986 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:31.275002 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:31.274911 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:31.379117 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:31.379078 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:37.717585 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:37.717546 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:37.839148 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:37.839111 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:44.752290 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:44.752239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:45.089973 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:45.089875 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:45.703454 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:45.703422 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:47.750852 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:47.750818 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:47.868855 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:47.868817 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:47.969940 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:47.969902 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:48.137274 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:48.137126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:48.551940 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:48.551901 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:48.651593 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:48.651556 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:52.300087 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:52.300046 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:54.600902 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:54.600852 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:33:58.848036 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.847989 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf"] Jul 23 18:33:58.848610 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.848585 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f7348d9-4915-4d8e-b00b-2a56714dc0d3" containerName="main" Jul 23 18:33:58.848665 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.848614 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f7348d9-4915-4d8e-b00b-2a56714dc0d3" containerName="main" Jul 23 18:33:58.848665 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.848655 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" containerName="main" Jul 23 18:33:58.848748 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.848665 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" containerName="main" Jul 23 18:33:58.848804 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.848789 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e0e4b65-5002-40ae-bfaa-eeaabd74c46e" containerName="main" Jul 23 18:33:58.848843 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.848807 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f7348d9-4915-4d8e-b00b-2a56714dc0d3" containerName="main" Jul 23 18:33:58.852628 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.852598 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:58.855311 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.855278 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-08efc605\"/\"openshift-service-ca.crt\"" Jul 23 18:33:58.856207 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.856151 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-08efc605\"/\"e2e-lifecycle-model-08efc605-kserve-self-signed-certs\"" Jul 23 18:33:58.856207 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.856200 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-08efc605\"/\"default-dockercfg-9ch4b\"" Jul 23 18:33:58.856459 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.856293 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-08efc605\"/\"kube-root-ca.crt\"" Jul 23 18:33:58.860355 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.860330 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf"] Jul 23 18:33:58.888323 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.888275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:58.970367 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.970329 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-home\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:58.970367 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.970367 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-model-cache\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:58.970648 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.970388 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tls-certs\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:58.970648 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.970414 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-dshm\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:58.970648 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.970514 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jzh5\" (UniqueName: \"kubernetes.io/projected/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-kube-api-access-9jzh5\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:58.970648 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:58.970621 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tmp-dir\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.071958 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.071914 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-home\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.071958 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.071960 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-model-cache\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.072207 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.072000 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tls-certs\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.072207 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.072028 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-dshm\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.072207 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.072064 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jzh5\" (UniqueName: \"kubernetes.io/projected/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-kube-api-access-9jzh5\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.072207 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.072138 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tmp-dir\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.072502 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.072477 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-home\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.072547 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.072501 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-model-cache\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.072603 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.072586 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tmp-dir\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.074479 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.074449 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-dshm\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.074752 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.074735 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tls-certs\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.081470 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.081443 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jzh5\" (UniqueName: \"kubernetes.io/projected/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-kube-api-access-9jzh5\") pod \"e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.165308 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.165170 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:33:59.308999 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:33:59.308957 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf"] Jul 23 18:33:59.312580 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:33:59.312550 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c428b4a_eaf9_4245_bc55_b7a5f0ca8509.slice/crio-7e2b94351f89191798d39c99b151ad2fd096befe3d9587f9996e916f1d9c1f35 WatchSource:0}: Error finding container 7e2b94351f89191798d39c99b151ad2fd096befe3d9587f9996e916f1d9c1f35: Status 404 returned error can't find the container with id 7e2b94351f89191798d39c99b151ad2fd096befe3d9587f9996e916f1d9c1f35 Jul 23 18:34:00.060916 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:00.060875 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" event={"ID":"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509","Type":"ContainerStarted","Data":"cb3af089d05ac626b0c97d70d01b6d04cfbb1090cf3a31f873b4ce5a26c3887b"} Jul 23 18:34:00.060916 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:00.060924 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" event={"ID":"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509","Type":"ContainerStarted","Data":"7e2b94351f89191798d39c99b151ad2fd096befe3d9587f9996e916f1d9c1f35"} Jul 23 18:34:00.083261 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:00.083176 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" podStartSLOduration=2.08314961 podStartE2EDuration="2.08314961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:33:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:34:00.08159675 +0000 UTC m=+2061.569088743" watchObservedRunningTime="2026-07-23 18:34:00.08314961 +0000 UTC m=+2061.570641602" Jul 23 18:34:02.967641 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:02.967545 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:34:09.166046 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:09.165997 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:34:09.166046 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:09.166059 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:34:09.179127 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:09.179096 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:34:10.128072 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:10.128026 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:34:17.169654 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:17.169601 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:34:28.483201 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:28.483162 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:34.553749 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:34.553705 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:34:36.247622 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:36.247575 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:39.124633 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:39.124604 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:34:39.131020 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:39.130993 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:34:40.803667 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:40.803627 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:43.070836 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:43.070788 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:44.955997 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:44.955949 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:34:48.152366 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:48.152324 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf"] Jul 23 18:34:48.249257 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:34:48.249191 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:03.173780 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.173735 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" podUID="5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" containerName="main" containerID="cri-o://cb3af089d05ac626b0c97d70d01b6d04cfbb1090cf3a31f873b4ce5a26c3887b" gracePeriod=45 Jul 23 18:35:03.398855 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.398818 2562 generic.go:356] "Generic (PLEG): container finished" podID="5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" containerID="cb3af089d05ac626b0c97d70d01b6d04cfbb1090cf3a31f873b4ce5a26c3887b" exitCode=0 Jul 23 18:35:03.399037 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.398894 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" event={"ID":"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509","Type":"ContainerDied","Data":"cb3af089d05ac626b0c97d70d01b6d04cfbb1090cf3a31f873b4ce5a26c3887b"} Jul 23 18:35:03.443203 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.443172 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:35:03.480820 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.480784 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-home\") pod \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " Jul 23 18:35:03.481016 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.480834 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tls-certs\") pod \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " Jul 23 18:35:03.481016 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.480859 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-model-cache\") pod \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " Jul 23 18:35:03.481141 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.481011 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tmp-dir\") pod \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " Jul 23 18:35:03.481141 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.481068 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jzh5\" (UniqueName: \"kubernetes.io/projected/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-kube-api-access-9jzh5\") pod \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " Jul 23 18:35:03.481315 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.481144 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-model-cache" (OuterVolumeSpecName: "model-cache") pod "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" (UID: "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:35:03.481315 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.481156 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-home" (OuterVolumeSpecName: "home") pod "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" (UID: "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:35:03.481315 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.481161 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-dshm\") pod \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\" (UID: \"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509\") " Jul 23 18:35:03.481315 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.481257 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" (UID: "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:35:03.481562 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.481541 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-home\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:35:03.481604 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.481567 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-model-cache\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:35:03.481604 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.481583 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tmp-dir\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:35:03.483908 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.483874 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-dshm" (OuterVolumeSpecName: "dshm") pod "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" (UID: "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:35:03.484061 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.483941 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" (UID: "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:35:03.484061 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.483991 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-kube-api-access-9jzh5" (OuterVolumeSpecName: "kube-api-access-9jzh5") pod "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" (UID: "5c428b4a-eaf9-4245-bc55-b7a5f0ca8509"). InnerVolumeSpecName "kube-api-access-9jzh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:35:03.582175 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.582132 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-dshm\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:35:03.582175 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.582167 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-tls-certs\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:35:03.582175 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:03.582178 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jzh5\" (UniqueName: \"kubernetes.io/projected/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509-kube-api-access-9jzh5\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:35:04.405459 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:04.405417 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" Jul 23 18:35:04.406026 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:04.405419 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf" event={"ID":"5c428b4a-eaf9-4245-bc55-b7a5f0ca8509","Type":"ContainerDied","Data":"7e2b94351f89191798d39c99b151ad2fd096befe3d9587f9996e916f1d9c1f35"} Jul 23 18:35:04.406026 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:04.405548 2562 scope.go:117] "RemoveContainer" containerID="cb3af089d05ac626b0c97d70d01b6d04cfbb1090cf3a31f873b4ce5a26c3887b" Jul 23 18:35:04.427530 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:04.427498 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf"] Jul 23 18:35:04.431806 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:04.431773 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-08efc605/e2e-lifecycle-model-08efc605-kserve-54c67f5cf5-cgvcf"] Jul 23 18:35:05.136955 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:05.136916 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" path="/var/lib/kubelet/pods/5c428b4a-eaf9-4245-bc55-b7a5f0ca8509/volumes" Jul 23 18:35:24.671952 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:24.671903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:25.313400 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:25.313352 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:31.178938 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:31.178830 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:31.279731 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:31.279692 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:31.451788 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:31.451749 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:31.742932 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:31.742827 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:32.814713 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:32.814673 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:32.912182 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:32.912136 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:33.008751 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:33.008708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:33.736431 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:33.736376 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:33.835340 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:33.835297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:33.932751 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:33.932712 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:35.331347 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:35.331302 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:35.354928 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:35.354886 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:35:35.787390 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:35.787354 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:35.886082 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:35.886041 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:41.389715 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:41.389656 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:41.490480 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:41.490434 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:42.081825 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:42.081747 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:45.056423 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:45.056384 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:35:48.744451 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:48.744415 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:48.852031 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:48.851994 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:48.945093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:48.945059 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:59.233573 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:59.233534 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:59.465318 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:59.465279 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:59.841313 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:35:59.841267 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:20.855232 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:20.855168 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:25.231493 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:25.231451 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:25.314644 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:25.314592 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:27.465290 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:27.465257 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:27.528058 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:27.528028 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:27.844544 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:27.844455 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:28.170842 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:28.170733 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:28.409363 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:28.409316 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:31.623170 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:31.623126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:31.710488 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:31.710449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:31.778916 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:31.778875 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:32.331814 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:32.331776 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:32.439145 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:32.439105 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:33.014531 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:33.014469 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:36.177009 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:36.176971 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:36.352022 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:36.351973 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:36:36.537577 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:36.537543 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:36.861083 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:36.860988 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:37.189245 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:37.189144 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:39.674908 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:39.674870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:39.780098 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:39.780057 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:39.916597 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:39.916562 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:40.117632 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:40.117596 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:40.462119 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:40.462083 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:40.563388 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:40.563353 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:44.180765 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:44.180728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:45.560002 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:45.559962 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:36:48.949814 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:48.949777 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7"] Jul 23 18:36:48.950293 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:48.950271 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" containerName="main" Jul 23 18:36:48.950293 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:48.950291 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" containerName="main" Jul 23 18:36:48.950854 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:48.950367 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c428b4a-eaf9-4245-bc55-b7a5f0ca8509" containerName="main" Jul 23 18:36:48.953464 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:48.953445 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:48.956757 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:48.956734 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-816eca\"/\"default-dockercfg-6svzk\"" Jul 23 18:36:48.956891 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:48.956785 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-816eca\"/\"kube-root-ca.crt\"" Jul 23 18:36:48.956891 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:48.956841 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-816eca\"/\"openshift-service-ca.crt\"" Jul 23 18:36:48.957016 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:48.956931 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-816eca\"/\"test-model-816eca-kserve-self-signed-certs\"" Jul 23 18:36:48.964781 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:48.964751 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7"] Jul 23 18:36:48.991317 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:48.991290 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:49.033842 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.033808 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-home\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.034001 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.033854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-model-cache\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.034001 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.033882 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-tmp-dir\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.034080 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.034043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-dshm\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.034080 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.034068 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa9bec7-73b6-4da8-be71-abb00502fe46-tls-certs\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.034146 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.034089 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x9g7\" (UniqueName: \"kubernetes.io/projected/dfa9bec7-73b6-4da8-be71-abb00502fe46-kube-api-access-4x9g7\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.134792 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.134743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-dshm\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.134792 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.134794 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa9bec7-73b6-4da8-be71-abb00502fe46-tls-certs\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.135049 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.134820 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x9g7\" (UniqueName: \"kubernetes.io/projected/dfa9bec7-73b6-4da8-be71-abb00502fe46-kube-api-access-4x9g7\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.135049 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.134844 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-home\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.135049 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.134872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-model-cache\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.135049 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.134902 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-tmp-dir\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.135298 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.135277 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-model-cache\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.135353 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.135277 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-home\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.135353 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.135323 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-tmp-dir\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.137139 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.137114 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-dshm\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.137267 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.137250 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa9bec7-73b6-4da8-be71-abb00502fe46-tls-certs\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.145149 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.145122 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x9g7\" (UniqueName: \"kubernetes.io/projected/dfa9bec7-73b6-4da8-be71-abb00502fe46-kube-api-access-4x9g7\") pod \"test-model-816eca-kserve-947df5c7c-d58q7\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.264471 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.264386 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:49.400356 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.400327 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7"] Jul 23 18:36:49.402605 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:36:49.402575 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfa9bec7_73b6_4da8_be71_abb00502fe46.slice/crio-a68a132b2930c50e3c2a50de752da4e6b7392f835dc4ed42d294fdc788c92029 WatchSource:0}: Error finding container a68a132b2930c50e3c2a50de752da4e6b7392f835dc4ed42d294fdc788c92029: Status 404 returned error can't find the container with id a68a132b2930c50e3c2a50de752da4e6b7392f835dc4ed42d294fdc788c92029 Jul 23 18:36:49.919650 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.919612 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" event={"ID":"dfa9bec7-73b6-4da8-be71-abb00502fe46","Type":"ContainerStarted","Data":"da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c"} Jul 23 18:36:49.919829 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.919660 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" event={"ID":"dfa9bec7-73b6-4da8-be71-abb00502fe46","Type":"ContainerStarted","Data":"a68a132b2930c50e3c2a50de752da4e6b7392f835dc4ed42d294fdc788c92029"} Jul 23 18:36:49.940856 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:49.940800 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" podStartSLOduration=1.940783258 podStartE2EDuration="1.940783258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:36:49.939011609 +0000 UTC m=+2231.426503614" watchObservedRunningTime="2026-07-23 18:36:49.940783258 +0000 UTC m=+2231.428275250" Jul 23 18:36:53.083584 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:53.083536 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:53.183587 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:53.183550 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:55.563474 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:55.563438 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:36:59.266989 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:59.266706 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:59.267715 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:59.267045 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:59.279450 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:59.279415 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:36:59.994946 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:36:59.994913 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:37:02.119413 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:02.119322 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:02.218490 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:02.218446 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:38.064105 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.064067 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4"] Jul 23 18:37:38.067876 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.067850 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.070264 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.070238 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-629a5d\"/\"openshift-service-ca.crt\"" Jul 23 18:37:38.071162 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.071138 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-629a5d\"/\"default-dockercfg-zlq6d\"" Jul 23 18:37:38.071162 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.071139 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-629a5d\"/\"kube-root-ca.crt\"" Jul 23 18:37:38.071466 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.071446 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-629a5d\"/\"test-model-629a5d-kserve-self-signed-certs\"" Jul 23 18:37:38.077861 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.077831 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4"] Jul 23 18:37:38.087537 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.087506 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:38.194856 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.194818 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljx58\" (UniqueName: \"kubernetes.io/projected/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-kube-api-access-ljx58\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.195047 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.194864 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-home\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.195047 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.194959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-model-cache\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.195047 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.195018 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tls-certs\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.195159 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.195051 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tmp-dir\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.195159 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.195071 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-dshm\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.296461 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.296421 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-model-cache\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.296647 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.296493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tls-certs\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.296647 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.296523 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tmp-dir\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.296647 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.296540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-dshm\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.296647 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.296570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljx58\" (UniqueName: \"kubernetes.io/projected/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-kube-api-access-ljx58\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.296840 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.296661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-home\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.296924 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.296900 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-model-cache\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.296966 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.296936 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tmp-dir\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.297018 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.296999 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-home\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.298948 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.298929 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-dshm\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.299120 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.299104 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tls-certs\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.305575 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.305551 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljx58\" (UniqueName: \"kubernetes.io/projected/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-kube-api-access-ljx58\") pod \"test-model-629a5d-kserve-6787b6cbd7-frzc4\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.379076 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.378966 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:38.522877 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:38.522801 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4"] Jul 23 18:37:38.525477 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:37:38.525443 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod340c7a5a_2fbd_4535_b3b9_8c691797fe5c.slice/crio-3c078e59fe8a2c1d6bb90a4d72e72ab1856cdbc29b3f007d65ed0fc877ea394d WatchSource:0}: Error finding container 3c078e59fe8a2c1d6bb90a4d72e72ab1856cdbc29b3f007d65ed0fc877ea394d: Status 404 returned error can't find the container with id 3c078e59fe8a2c1d6bb90a4d72e72ab1856cdbc29b3f007d65ed0fc877ea394d Jul 23 18:37:39.173575 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:39.173535 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" event={"ID":"340c7a5a-2fbd-4535-b3b9-8c691797fe5c","Type":"ContainerStarted","Data":"5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa"} Jul 23 18:37:39.173939 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:39.173584 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" event={"ID":"340c7a5a-2fbd-4535-b3b9-8c691797fe5c","Type":"ContainerStarted","Data":"3c078e59fe8a2c1d6bb90a4d72e72ab1856cdbc29b3f007d65ed0fc877ea394d"} Jul 23 18:37:39.415045 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:39.415009 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:42.056642 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:42.056578 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" podStartSLOduration=4.056558445 podStartE2EDuration="4.056558445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:37:39.201304036 +0000 UTC m=+2280.688796019" watchObservedRunningTime="2026-07-23 18:37:42.056558445 +0000 UTC m=+2283.544050436" Jul 23 18:37:42.057358 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:42.057332 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:37:48.379201 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:48.379159 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:48.379722 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:48.379241 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:48.391937 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:48.391910 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:49.238771 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:49.238737 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:37:56.248693 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:37:56.248650 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:38:19.534347 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:19.534302 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7"] Jul 23 18:38:19.557799 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:19.557761 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:20.533091 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:20.533055 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4"] Jul 23 18:38:20.548867 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:20.548824 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:21.300445 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:21.300403 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:21.400849 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:21.400805 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:24.557635 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:24.557595 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:38:31.259062 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:31.258955 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:34.556027 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.555967 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" podUID="dfa9bec7-73b6-4da8-be71-abb00502fe46" containerName="main" containerID="cri-o://da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c" gracePeriod=45 Jul 23 18:38:34.821606 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.821523 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:38:34.937616 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.937581 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-model-cache\") pod \"dfa9bec7-73b6-4da8-be71-abb00502fe46\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " Jul 23 18:38:34.937616 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.937627 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-dshm\") pod \"dfa9bec7-73b6-4da8-be71-abb00502fe46\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " Jul 23 18:38:34.937905 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.937663 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-home\") pod \"dfa9bec7-73b6-4da8-be71-abb00502fe46\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " Jul 23 18:38:34.937905 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.937737 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa9bec7-73b6-4da8-be71-abb00502fe46-tls-certs\") pod \"dfa9bec7-73b6-4da8-be71-abb00502fe46\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " Jul 23 18:38:34.937905 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.937763 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4x9g7\" (UniqueName: \"kubernetes.io/projected/dfa9bec7-73b6-4da8-be71-abb00502fe46-kube-api-access-4x9g7\") pod \"dfa9bec7-73b6-4da8-be71-abb00502fe46\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " Jul 23 18:38:34.937905 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.937832 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-tmp-dir\") pod \"dfa9bec7-73b6-4da8-be71-abb00502fe46\" (UID: \"dfa9bec7-73b6-4da8-be71-abb00502fe46\") " Jul 23 18:38:34.938133 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.937909 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-model-cache" (OuterVolumeSpecName: "model-cache") pod "dfa9bec7-73b6-4da8-be71-abb00502fe46" (UID: "dfa9bec7-73b6-4da8-be71-abb00502fe46"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:38:34.938133 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.938052 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-home" (OuterVolumeSpecName: "home") pod "dfa9bec7-73b6-4da8-be71-abb00502fe46" (UID: "dfa9bec7-73b6-4da8-be71-abb00502fe46"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:38:34.938133 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.938109 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dfa9bec7-73b6-4da8-be71-abb00502fe46" (UID: "dfa9bec7-73b6-4da8-be71-abb00502fe46"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:38:34.938302 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.938206 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-tmp-dir\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:34.938302 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.938261 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-model-cache\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:34.938302 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.938274 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-home\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:34.940436 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.940399 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-dshm" (OuterVolumeSpecName: "dshm") pod "dfa9bec7-73b6-4da8-be71-abb00502fe46" (UID: "dfa9bec7-73b6-4da8-be71-abb00502fe46"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:38:34.940436 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.940402 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfa9bec7-73b6-4da8-be71-abb00502fe46-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dfa9bec7-73b6-4da8-be71-abb00502fe46" (UID: "dfa9bec7-73b6-4da8-be71-abb00502fe46"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:38:34.940607 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:34.940516 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa9bec7-73b6-4da8-be71-abb00502fe46-kube-api-access-4x9g7" (OuterVolumeSpecName: "kube-api-access-4x9g7") pod "dfa9bec7-73b6-4da8-be71-abb00502fe46" (UID: "dfa9bec7-73b6-4da8-be71-abb00502fe46"). InnerVolumeSpecName "kube-api-access-4x9g7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:38:35.039166 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.039110 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa9bec7-73b6-4da8-be71-abb00502fe46-tls-certs\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:35.039166 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.039155 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x9g7\" (UniqueName: \"kubernetes.io/projected/dfa9bec7-73b6-4da8-be71-abb00502fe46-kube-api-access-4x9g7\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:35.039166 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.039170 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa9bec7-73b6-4da8-be71-abb00502fe46-dshm\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:35.462381 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.462336 2562 generic.go:356] "Generic (PLEG): container finished" podID="dfa9bec7-73b6-4da8-be71-abb00502fe46" containerID="da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c" exitCode=0 Jul 23 18:38:35.462601 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.462394 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" Jul 23 18:38:35.462601 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.462419 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" event={"ID":"dfa9bec7-73b6-4da8-be71-abb00502fe46","Type":"ContainerDied","Data":"da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c"} Jul 23 18:38:35.462601 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.462468 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7" event={"ID":"dfa9bec7-73b6-4da8-be71-abb00502fe46","Type":"ContainerDied","Data":"a68a132b2930c50e3c2a50de752da4e6b7392f835dc4ed42d294fdc788c92029"} Jul 23 18:38:35.462601 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.462489 2562 scope.go:117] "RemoveContainer" containerID="da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c" Jul 23 18:38:35.471915 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.471889 2562 scope.go:117] "RemoveContainer" containerID="da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c" Jul 23 18:38:35.472289 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:38:35.472260 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c\": container with ID starting with da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c not found: ID does not exist" containerID="da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c" Jul 23 18:38:35.472403 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.472296 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c"} err="failed to get container status \"da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c\": rpc error: code = NotFound desc = could not find container \"da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c\": container with ID starting with da8e1b85ff1ad09a322c55fc528c8d75f3d6e87f359e0601e041983a585b126c not found: ID does not exist" Jul 23 18:38:35.482616 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.482573 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7"] Jul 23 18:38:35.489502 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.489468 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-816eca/test-model-816eca-kserve-947df5c7c-d58q7"] Jul 23 18:38:35.554394 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.554347 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" podUID="340c7a5a-2fbd-4535-b3b9-8c691797fe5c" containerName="main" containerID="cri-o://5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa" gracePeriod=45 Jul 23 18:38:35.752546 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.752364 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:35.826009 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.825976 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:38:35.947863 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.947819 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-model-cache\") pod \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " Jul 23 18:38:35.947863 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.947870 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljx58\" (UniqueName: \"kubernetes.io/projected/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-kube-api-access-ljx58\") pod \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " Jul 23 18:38:35.948124 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.947931 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tls-certs\") pod \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " Jul 23 18:38:35.948124 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.947974 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-home\") pod \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " Jul 23 18:38:35.948124 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.948012 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tmp-dir\") pod \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " Jul 23 18:38:35.948124 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.948100 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-dshm\") pod \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\" (UID: \"340c7a5a-2fbd-4535-b3b9-8c691797fe5c\") " Jul 23 18:38:35.948337 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.948174 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-model-cache" (OuterVolumeSpecName: "model-cache") pod "340c7a5a-2fbd-4535-b3b9-8c691797fe5c" (UID: "340c7a5a-2fbd-4535-b3b9-8c691797fe5c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:38:35.948382 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.948363 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-home" (OuterVolumeSpecName: "home") pod "340c7a5a-2fbd-4535-b3b9-8c691797fe5c" (UID: "340c7a5a-2fbd-4535-b3b9-8c691797fe5c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:38:35.948429 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.948389 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "340c7a5a-2fbd-4535-b3b9-8c691797fe5c" (UID: "340c7a5a-2fbd-4535-b3b9-8c691797fe5c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:38:35.948541 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.948525 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-home\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:35.948600 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.948542 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tmp-dir\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:35.948600 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.948554 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-model-cache\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:35.950618 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.950585 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-dshm" (OuterVolumeSpecName: "dshm") pod "340c7a5a-2fbd-4535-b3b9-8c691797fe5c" (UID: "340c7a5a-2fbd-4535-b3b9-8c691797fe5c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:38:35.950816 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.950684 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "340c7a5a-2fbd-4535-b3b9-8c691797fe5c" (UID: "340c7a5a-2fbd-4535-b3b9-8c691797fe5c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:38:35.950816 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:35.950723 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-kube-api-access-ljx58" (OuterVolumeSpecName: "kube-api-access-ljx58") pod "340c7a5a-2fbd-4535-b3b9-8c691797fe5c" (UID: "340c7a5a-2fbd-4535-b3b9-8c691797fe5c"). InnerVolumeSpecName "kube-api-access-ljx58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:38:36.049527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.049417 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-tls-certs\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:36.049527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.049464 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-dshm\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:36.049527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.049478 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljx58\" (UniqueName: \"kubernetes.io/projected/340c7a5a-2fbd-4535-b3b9-8c691797fe5c-kube-api-access-ljx58\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:38:36.364620 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.364507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:36.468922 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.468889 2562 generic.go:356] "Generic (PLEG): container finished" podID="340c7a5a-2fbd-4535-b3b9-8c691797fe5c" containerID="5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa" exitCode=0 Jul 23 18:38:36.469133 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.468931 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" Jul 23 18:38:36.469133 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.468969 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" event={"ID":"340c7a5a-2fbd-4535-b3b9-8c691797fe5c","Type":"ContainerDied","Data":"5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa"} Jul 23 18:38:36.469133 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.469016 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4" event={"ID":"340c7a5a-2fbd-4535-b3b9-8c691797fe5c","Type":"ContainerDied","Data":"3c078e59fe8a2c1d6bb90a4d72e72ab1856cdbc29b3f007d65ed0fc877ea394d"} Jul 23 18:38:36.469133 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.469036 2562 scope.go:117] "RemoveContainer" containerID="5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa" Jul 23 18:38:36.479066 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.479045 2562 scope.go:117] "RemoveContainer" containerID="5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa" Jul 23 18:38:36.479470 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:38:36.479450 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa\": container with ID starting with 5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa not found: ID does not exist" containerID="5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa" Jul 23 18:38:36.479545 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.479480 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa"} err="failed to get container status \"5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa\": rpc error: code = NotFound desc = could not find container \"5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa\": container with ID starting with 5508cee950b820e18c235657e70b7c492e8b5763526b29fb3724b0c427c439fa not found: ID does not exist" Jul 23 18:38:36.491659 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.491610 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4"] Jul 23 18:38:36.495458 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:36.495421 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-629a5d/test-model-629a5d-kserve-6787b6cbd7-frzc4"] Jul 23 18:38:37.131828 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:37.131787 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="340c7a5a-2fbd-4535-b3b9-8c691797fe5c" path="/var/lib/kubelet/pods/340c7a5a-2fbd-4535-b3b9-8c691797fe5c/volumes" Jul 23 18:38:37.132261 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:37.132246 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfa9bec7-73b6-4da8-be71-abb00502fe46" path="/var/lib/kubelet/pods/dfa9bec7-73b6-4da8-be71-abb00502fe46/volumes" Jul 23 18:38:37.150278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:37.150234 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:38:42.049164 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:42.049121 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:42.172203 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:42.172156 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:42.387090 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:42.386984 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:42.621625 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:42.621590 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:43.463315 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:43.463281 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:43.561029 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:43.560980 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:53.172759 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:53.172720 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:55.756659 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:55.756623 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:38:57.835442 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:57.835396 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:58.240788 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:38:58.240746 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:02.798341 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:02.798297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:04.131561 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:04.131524 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:04.563535 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:04.563472 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:04.627003 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:04.626953 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:04.690322 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:04.690285 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:05.040498 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:05.040452 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:05.365107 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:05.364994 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:05.475742 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:05.475699 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:08.428159 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:08.428110 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:08.522155 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:08.522120 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:08.606280 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:08.606242 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:08.710232 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:08.710149 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:09.115284 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:09.115169 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:09.215118 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:09.215078 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:11.347930 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:11.347888 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:39:12.899238 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:12.899191 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:13.436650 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:13.436590 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:13.763299 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:13.763260 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:13.896867 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:13.896823 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:16.418616 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:16.418556 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:16.505931 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:16.505893 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:16.567034 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:16.566981 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:16.667045 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:16.667005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:17.218551 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:17.218507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:17.339569 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:17.339518 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:20.950951 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:20.950904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:23.663586 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:23.663547 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:39:29.836415 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:29.836373 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:29.936964 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:29.936929 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:38.021534 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:38.021463 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:38.122266 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:38.122225 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:39.155444 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:39.155414 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:39:39.164408 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:39.164384 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:39:39.759406 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:39.759363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:39.858553 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:39.858514 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:45.017301 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:45.017233 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:49.272694 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:49.272642 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:49.714898 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:49.714859 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:51.564115 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:51.564077 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:39:55.779943 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:55.779909 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:55.847584 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:55.847536 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:56.085696 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:56.085607 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:56.324595 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:56.324554 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:56.998460 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:56.998415 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:57.097720 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:39:57.097678 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:11.976416 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:11.976316 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:16.452812 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:16.452778 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:40:16.530187 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:16.530146 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:16.959144 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:16.959095 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:23.427756 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:23.427714 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:23.493876 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:23.493840 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:23.739422 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:23.739376 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:24.005734 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:24.005624 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:24.404425 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:24.404329 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:24.766607 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:24.766526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:25.188764 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:25.188662 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:27.820761 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:27.820711 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:27.915638 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:27.915587 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:28.093053 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:28.092944 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:28.188082 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:28.188043 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:28.713671 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:28.713632 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:28.816558 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:28.816517 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:32.404434 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:32.404367 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:32.792386 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:32.792342 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:33.126642 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:33.126524 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:33.481939 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:33.481900 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:35.068488 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:35.068436 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:40:35.870504 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:35.870456 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:35.978417 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:35.978368 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:36.074799 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:36.074756 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:36.174366 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:36.174268 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:36.654533 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:36.654496 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:36.754583 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:36.754533 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:37.491201 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:37.491154 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:37.639326 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:37.639288 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:38.653654 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:38.653610 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:38.753621 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:38.753579 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:39.468178 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:39.468140 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:39.566831 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:39.566787 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:40.435283 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:40.435239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:40.531626 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:40.531590 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:40.632618 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:40.632575 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:41.270852 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:41.270813 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:41.674381 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:41.674206 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:42.201245 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:42.201167 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:42.305254 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:42.305189 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:43.001402 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:43.001359 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:43.102056 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:43.102020 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:43.986479 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:43.986409 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:44.085963 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:44.085911 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:45.294652 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:45.294577 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:45.391952 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:45.391904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:45.733393 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:45.733356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:45.840931 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:45.840889 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:46.458679 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:46.458644 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:40:46.520832 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:46.520793 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:46.622924 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:46.622886 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:47.464845 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:47.464806 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:47.561900 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:47.561855 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:48.258757 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:48.258722 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:48.360770 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:48.360733 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:49.292113 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:49.292081 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:49.392559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:49.392518 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:50.156652 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:50.156612 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:50.245137 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:50.245100 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:51.399754 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:51.399709 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:51.507856 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:51.507816 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:52.296432 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:52.296384 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:52.391723 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:52.391687 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:53.461835 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:53.461798 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:53.584956 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:53.584914 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:54.511790 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:54.511752 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:54.612370 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:54.612324 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:55.465763 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:55.465732 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:55.568974 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:55.568930 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:56.258708 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.258669 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:56.362900 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.362854 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:56.966338 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.966300 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn"] Jul 23 18:40:56.966791 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.966688 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="340c7a5a-2fbd-4535-b3b9-8c691797fe5c" containerName="main" Jul 23 18:40:56.966791 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.966699 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="340c7a5a-2fbd-4535-b3b9-8c691797fe5c" containerName="main" Jul 23 18:40:56.966791 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.966719 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfa9bec7-73b6-4da8-be71-abb00502fe46" containerName="main" Jul 23 18:40:56.966791 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.966727 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfa9bec7-73b6-4da8-be71-abb00502fe46" containerName="main" Jul 23 18:40:56.966941 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.966821 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="340c7a5a-2fbd-4535-b3b9-8c691797fe5c" containerName="main" Jul 23 18:40:56.966941 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.966835 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfa9bec7-73b6-4da8-be71-abb00502fe46" containerName="main" Jul 23 18:40:56.970276 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.970254 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:56.973512 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.973481 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"openshift-service-ca.crt\"" Jul 23 18:40:56.973737 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.973500 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"default-dockercfg-zfz4b\"" Jul 23 18:40:56.973869 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.973501 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"auth-test-model-5f06b1-kserve-self-signed-certs\"" Jul 23 18:40:56.973869 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.973524 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"kube-root-ca.crt\"" Jul 23 18:40:56.981627 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.981555 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn"] Jul 23 18:40:56.985078 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:56.985047 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:57.048145 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.048105 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbzwg\" (UniqueName: \"kubernetes.io/projected/893249a5-2e3f-4f76-9833-f22c2e6c56f9-kube-api-access-hbzwg\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.048412 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.048200 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-model-cache\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.048412 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.048306 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tmp-dir\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.048412 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.048328 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-home\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.048412 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.048376 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tls-certs\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.048412 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.048394 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-dshm\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.149201 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.149161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tls-certs\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.149201 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.149201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-dshm\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.149543 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.149289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbzwg\" (UniqueName: \"kubernetes.io/projected/893249a5-2e3f-4f76-9833-f22c2e6c56f9-kube-api-access-hbzwg\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.149543 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.149334 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-model-cache\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.149543 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.149389 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tmp-dir\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.149543 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.149406 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-home\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.149946 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.149797 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-model-cache\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.149946 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.149819 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tmp-dir\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.149946 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.149855 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-home\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.151924 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.151896 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-dshm\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.152099 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.152082 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tls-certs\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.160278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.160243 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbzwg\" (UniqueName: \"kubernetes.io/projected/893249a5-2e3f-4f76-9833-f22c2e6c56f9-kube-api-access-hbzwg\") pod \"auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.266016 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.265911 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:57.282464 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.282420 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:40:57.366355 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.366317 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:57.431053 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:57.431026 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn"] Jul 23 18:40:57.433638 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:40:57.433597 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod893249a5_2e3f_4f76_9833_f22c2e6c56f9.slice/crio-04fd028cb0fba8293169f94d9e566bb7a0a6a087de6d03d86256f49a621235d9 WatchSource:0}: Error finding container 04fd028cb0fba8293169f94d9e566bb7a0a6a087de6d03d86256f49a621235d9: Status 404 returned error can't find the container with id 04fd028cb0fba8293169f94d9e566bb7a0a6a087de6d03d86256f49a621235d9 Jul 23 18:40:58.044536 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:58.044490 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:58.145279 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:58.145244 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:58.168891 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:58.168852 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" event={"ID":"893249a5-2e3f-4f76-9833-f22c2e6c56f9","Type":"ContainerStarted","Data":"57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596"} Jul 23 18:40:58.168891 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:58.168898 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" event={"ID":"893249a5-2e3f-4f76-9833-f22c2e6c56f9","Type":"ContainerStarted","Data":"04fd028cb0fba8293169f94d9e566bb7a0a6a087de6d03d86256f49a621235d9"} Jul 23 18:40:58.186910 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:58.186854 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" podStartSLOduration=2.186834879 podStartE2EDuration="2.186834879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:40:58.1859041 +0000 UTC m=+2479.673396097" watchObservedRunningTime="2026-07-23 18:40:58.186834879 +0000 UTC m=+2479.674326870" Jul 23 18:40:58.844234 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:58.844179 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:58.943107 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:58.943066 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:59.653126 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:59.653084 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:59.752824 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:40:59.752767 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:00.680465 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:00.680414 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:00.755530 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:00.755491 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:41:00.757808 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:00.757779 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:01.536306 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:01.536261 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:01.634917 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:01.634869 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:03.620104 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:03.620060 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:03.721310 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:03.721276 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:04.589712 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:04.589674 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:04.688563 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:04.688529 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:05.142395 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:05.142336 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:06.732683 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:06.732644 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:06.832401 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:06.832363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:06.975485 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:06.975437 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:07.075387 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:07.075278 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:07.283498 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:07.283459 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:41:07.283724 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:07.283524 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:41:07.296426 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:07.296395 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:41:07.893805 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:07.893762 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:08.007585 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:08.007544 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:08.229917 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:08.229887 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:41:09.228110 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:09.228073 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:09.375746 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:09.375706 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:10.612430 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:10.612385 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:10.794270 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:10.794230 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:11.569102 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:11.569064 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:11.671388 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:11.671339 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:12.678817 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:12.678775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:12.779664 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:12.779621 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:13.539189 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:13.539146 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:13.639809 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:13.639770 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:14.374350 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:14.374294 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:14.474404 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:14.474342 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:15.365004 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:15.364959 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:15.464861 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:15.464826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:16.137384 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:16.137341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:16.238325 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:16.238281 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:17.097063 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:17.096473 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:17.202577 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:17.202532 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:17.972984 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:17.972942 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:18.073924 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.073881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:18.785899 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.785856 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:18.891255 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.891196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:18.891525 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.891504 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86"] Jul 23 18:41:18.895057 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.895035 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:18.899073 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.899042 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"openshift-service-ca.crt\"" Jul 23 18:41:18.899073 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.899069 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"kube-root-ca.crt\"" Jul 23 18:41:18.899301 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.899055 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"default-dockercfg-x4v54\"" Jul 23 18:41:18.899301 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.899103 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"auth-test-model-8ee985-kserve-self-signed-certs\"" Jul 23 18:41:18.916396 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.916360 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86"] Jul 23 18:41:18.952119 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.952083 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-home\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:18.952119 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.952121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40440bd9-635c-43ec-b3cc-77335a574826-tls-certs\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:18.952395 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.952142 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-tmp-dir\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:18.952395 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.952276 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-model-cache\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:18.952395 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.952332 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcqnk\" (UniqueName: \"kubernetes.io/projected/40440bd9-635c-43ec-b3cc-77335a574826-kube-api-access-gcqnk\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:18.952395 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.952367 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-dshm\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:18.966943 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:18.966904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:19.053475 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.053366 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-home\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.053475 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.053404 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40440bd9-635c-43ec-b3cc-77335a574826-tls-certs\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.053475 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.053424 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-tmp-dir\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.053475 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.053459 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-model-cache\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.053854 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.053489 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcqnk\" (UniqueName: \"kubernetes.io/projected/40440bd9-635c-43ec-b3cc-77335a574826-kube-api-access-gcqnk\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.053854 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.053510 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-dshm\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.053961 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.053889 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-home\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.053961 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.053917 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-model-cache\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.053961 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.053937 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-tmp-dir\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.056082 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.056056 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-dshm\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.056288 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.056269 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40440bd9-635c-43ec-b3cc-77335a574826-tls-certs\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.066566 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.066533 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcqnk\" (UniqueName: \"kubernetes.io/projected/40440bd9-635c-43ec-b3cc-77335a574826-kube-api-access-gcqnk\") pod \"auth-test-model-8ee985-kserve-59bbbb8f9-tlp86\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.206329 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.206286 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:19.358705 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.358677 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86"] Jul 23 18:41:19.360984 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:41:19.360952 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40440bd9_635c_43ec_b3cc_77335a574826.slice/crio-b96b17ee06b3fbd5a8472db08c85bc69aeebe2016e8006648a33aff4fcd67da7 WatchSource:0}: Error finding container b96b17ee06b3fbd5a8472db08c85bc69aeebe2016e8006648a33aff4fcd67da7: Status 404 returned error can't find the container with id b96b17ee06b3fbd5a8472db08c85bc69aeebe2016e8006648a33aff4fcd67da7 Jul 23 18:41:19.799256 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.799195 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:19.898433 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:19.898394 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:20.285240 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:20.285187 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" event={"ID":"40440bd9-635c-43ec-b3cc-77335a574826","Type":"ContainerStarted","Data":"34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a"} Jul 23 18:41:20.285426 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:20.285250 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" event={"ID":"40440bd9-635c-43ec-b3cc-77335a574826","Type":"ContainerStarted","Data":"b96b17ee06b3fbd5a8472db08c85bc69aeebe2016e8006648a33aff4fcd67da7"} Jul 23 18:41:20.306410 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:20.306350 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" podStartSLOduration=2.306329617 podStartE2EDuration="2.306329617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:41:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:41:20.304292246 +0000 UTC m=+2501.791784239" watchObservedRunningTime="2026-07-23 18:41:20.306329617 +0000 UTC m=+2501.793821610" Jul 23 18:41:20.931100 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:20.930963 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:21.031705 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:21.031665 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:22.574184 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:22.574141 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:41:22.577822 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:22.577789 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:22.675965 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:22.675906 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:22.889585 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:22.889489 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:22.992225 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:22.992170 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:23.889051 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:23.889010 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:24.023752 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:24.023714 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:24.988466 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:24.988406 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:25.088079 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:25.088036 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:25.931808 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:25.931762 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:26.705641 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:26.705583 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:27.119473 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:27.119366 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:27.240471 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:27.240429 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:27.993711 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:27.993660 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:28.094475 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:28.094426 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:28.888775 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:28.888719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:28.987022 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:28.986981 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:29.207071 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:29.207038 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:29.207633 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:29.207118 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:29.219883 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:29.219848 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:29.343123 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:29.343090 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:41:29.896858 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:29.896805 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:30.013195 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:30.013148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:31.036382 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:31.036288 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:31.160131 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:31.160093 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:32.009988 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:32.009950 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:32.108186 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:32.108148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:32.236727 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:32.236674 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:33.050682 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:33.050639 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:33.150010 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:33.149969 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:34.005915 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:34.005876 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:34.109588 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:34.109548 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:34.899086 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:34.899045 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:35.041174 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:35.041137 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:35.914282 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:35.914244 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:36.083605 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:36.083564 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:36.942138 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:36.942090 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:37.048352 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:37.048311 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:38.127283 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:38.127207 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:38.235074 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:38.235035 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:39.376762 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:39.376719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:39.519382 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:39.519328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:40.479397 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:40.479352 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:40.586346 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:40.586305 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:41.471440 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:41.471400 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:41.596154 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:41.596113 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:42.770259 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:42.770196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:42.897740 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:42.897701 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:43.828765 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:43.828723 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:41:44.253728 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:44.253676 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:44.358704 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:44.358661 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:45.416180 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:45.416135 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:45.718726 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:45.718687 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:46.868461 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:46.868417 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:46.997158 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:46.997121 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:48.177987 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:48.177945 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:48.371460 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:48.371423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:49.502742 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:49.502705 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:49.679394 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:49.679349 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:50.859553 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:50.859507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:51.031499 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:51.031461 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:52.231671 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:52.231629 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:52.359552 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:52.359502 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:53.388064 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:53.388024 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:53.560276 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:53.560239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:54.837417 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:54.837377 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:54.961015 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:54.960973 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:55.960132 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:55.960093 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:56.065576 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:56.065529 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:57.021855 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:57.021814 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:57.119600 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:57.119560 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:58.115453 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:58.115406 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:58.231044 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:58.231003 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:59.349096 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:59.349058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:59.477567 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:41:59.477524 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:00.798148 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:00.798106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:00.967638 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:00.967600 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:01.700615 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:01.700579 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:42:02.550768 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:02.550720 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:02.669102 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:02.669060 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:03.726914 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:03.726853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:03.897019 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:03.896974 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:04.660452 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:04.660411 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:05.539644 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:05.539594 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:05.761409 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:05.761370 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:06.889120 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:06.889079 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:07.038669 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:07.038627 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:08.663094 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:08.663050 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:08.830516 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:08.830478 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:10.046294 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:10.046248 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:10.325168 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:10.325067 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:11.525539 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:11.525502 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:11.691867 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:11.691819 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:13.279046 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:13.279003 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:13.379981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:13.379940 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:13.968393 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:13.968354 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:14.077498 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:14.077458 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:15.603091 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:15.603050 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:15.778404 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:15.778355 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:16.192032 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:16.191996 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:17.035272 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:17.035199 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:17.251774 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:17.251728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:18.444716 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:18.444679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:18.567813 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:18.567771 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:19.724269 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:19.724200 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:19.894999 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:19.894962 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:21.189330 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:21.189287 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:21.401294 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:21.401249 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:22.290400 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:22.290358 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:22.429514 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:22.429470 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:23.847358 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:23.847320 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:24.002992 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:24.002950 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:24.798144 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:24.798102 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:42:25.158397 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:25.158285 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:25.278976 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:25.278934 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:26.241488 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:26.241451 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:26.377083 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:26.377038 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:27.266204 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:27.266161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:27.381259 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:27.381190 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:28.301311 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:28.301268 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:28.401631 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:28.401596 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:29.269152 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:29.269113 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:29.393023 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:29.392975 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:30.528474 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:30.528432 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:30.736010 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:30.735974 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:31.331151 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:31.331119 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:31.432734 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:31.432694 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:32.275332 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:32.275298 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:32.374970 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:32.374938 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:33.545057 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:33.544941 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:33.669899 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:33.669854 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:34.669632 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:34.669592 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:34.789042 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:34.788998 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:35.623116 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:35.623076 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:35.731142 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:35.731109 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:36.771286 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:36.771253 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:36.872646 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:36.872598 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:37.573007 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:37.572970 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:37.670299 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:37.670255 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:38.264635 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:38.264600 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:42:38.663150 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:38.663044 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:38.758449 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:38.758412 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:39.562368 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:39.562326 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:41.437052 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:41.437015 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:41.533843 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:41.533808 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:42.431655 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:42.431612 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:42.532053 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:42.532008 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:43.329813 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:43.329776 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:43.429606 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:43.429565 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:44.184978 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:44.184942 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:44.285702 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:44.285655 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:45.170594 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:45.170563 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:45.272626 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:45.272592 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:45.976827 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:45.976787 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:46.078039 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:46.077997 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:46.812801 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:46.812759 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:46.912389 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:46.912349 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:47.897402 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:47.897364 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:48.019460 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:48.019419 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:48.907310 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:48.907273 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:49.010081 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:49.010038 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:50.026950 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:50.026907 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:50.129426 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:50.129387 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:51.107383 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:51.107345 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:51.207901 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:51.207865 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:51.308499 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:51.308458 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:51.954570 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:51.954537 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:52.055802 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:52.055767 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:52.885451 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:52.885408 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:52.984786 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:52.984744 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:53.823062 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:53.823027 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:53.923726 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:53.923692 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:54.833391 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:54.833351 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:54.933685 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:54.933653 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:55.661418 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:55.661379 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:55.762310 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:55.762191 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:56.777317 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:56.777265 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:56.778553 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:56.778528 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:42:56.873072 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:56.873031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:57.703812 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:57.703771 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:57.804673 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:57.804618 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:58.758918 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:58.758865 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:58.862990 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:58.862952 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:59.816437 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:59.816396 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:59.926024 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:42:59.925982 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:00.957578 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:00.957478 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:01.156551 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:01.156498 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:02.251296 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:02.251258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:02.344226 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:02.344179 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:03.345124 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:03.345086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:03.474690 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:03.474657 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:04.319722 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:04.319679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:04.444344 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:04.444301 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:05.336481 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:05.336444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:05.436342 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:05.436301 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:06.221356 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:06.221316 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:06.338173 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:06.338115 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:07.451464 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:07.451412 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:07.576106 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:07.576057 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:08.355192 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:08.355154 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:08.454647 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:08.454606 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:09.422288 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:09.422244 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:09.522261 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:09.522190 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:10.307868 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:10.307828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:10.409295 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:10.409240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:11.261737 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:11.261694 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:11.356867 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:11.356825 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:12.134264 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:12.134198 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:12.235197 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:12.235160 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:13.070697 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:13.070657 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:13.169088 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:13.169050 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:13.883063 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:13.883005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:13.983595 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:13.983557 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:14.907403 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:14.907363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:15.007892 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:15.007853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:15.811897 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:15.811861 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:15.912323 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:15.912283 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:16.748107 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:16.748070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:16.852186 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:16.852143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:16.868182 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:16.868144 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:43:18.026634 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:18.026568 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:18.174683 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:18.174639 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:19.217165 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:19.217119 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:19.323160 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:19.323106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:20.209572 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:20.209529 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:20.316631 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:20.316587 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:21.400107 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:21.400070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:21.499724 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:21.499682 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:22.261284 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:22.261243 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:22.361772 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:22.361724 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:23.293395 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:23.293359 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:23.394140 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:23.394099 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:24.171641 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:24.171579 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:24.270231 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:24.270185 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:25.281844 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:25.281804 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:25.381972 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:25.381933 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:26.181511 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:26.181478 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:26.282300 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:26.282261 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:27.157420 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:27.157385 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:27.259047 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:27.259009 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:27.976551 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:27.976447 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:28.082198 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:28.082148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:28.855073 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:28.855033 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:28.950108 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:28.950072 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:29.762581 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:29.762539 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:29.863867 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:29.863833 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:30.860947 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:30.860906 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:30.960156 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:30.960110 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:31.699864 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:31.699816 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:31.803088 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:31.803038 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:32.705629 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:32.705583 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:32.803278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:32.803207 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:33.579130 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:33.579085 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:33.683848 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:33.683812 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:34.416904 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:34.416866 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:34.514542 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:34.514506 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:35.645784 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:35.645749 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:35.747483 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:35.747439 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:36.587349 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:36.587307 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:36.688227 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:36.688170 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:36.869912 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:36.869808 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:43:37.578995 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:37.578955 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:37.675551 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:37.675490 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:38.627229 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:38.627169 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:38.728536 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:38.728496 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:39.656433 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:39.656394 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:39.764549 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:39.764510 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:40.539126 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:40.539092 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:40.640893 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:40.640849 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:41.433696 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:41.433655 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:41.534826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:41.534779 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:42.293014 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:42.292979 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn"] Jul 23 18:43:42.293406 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:42.293358 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:42.389413 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:42.389366 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:43.137991 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:43.137954 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:43.236592 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:43.236553 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:43.291693 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:43.291650 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86"] Jul 23 18:43:43.332621 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:43.332578 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:44.014719 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:44.014675 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:44.114747 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:44.114699 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:45.037434 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:45.037395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:45.138436 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:45.138387 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:45.846495 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:45.846457 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:45.937607 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:45.937571 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:46.778685 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:46.778647 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:46.877590 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:46.877548 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:47.781685 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:47.781641 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:47.882361 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:47.882314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:48.583453 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:48.583417 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:48.683328 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:48.683284 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:49.587702 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:49.587666 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:49.686497 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:49.686449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:50.444545 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:50.444502 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:50.541349 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:50.541313 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:51.314983 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:51.314940 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:51.415291 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:51.415253 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:52.126812 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:52.126780 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:52.227068 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:52.227026 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:53.175409 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:53.175368 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:53.277579 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:53.277540 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:53.972729 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:53.972687 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:54.072757 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:54.072720 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:54.919171 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:54.919128 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:55.018878 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:55.018845 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:55.119089 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:55.119052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:55.697028 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:55.696989 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:43:55.827736 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:55.827698 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:55.931760 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:55.931719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:56.736174 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:56.736140 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:56.836589 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:56.836549 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:57.314339 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.314296 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" podUID="893249a5-2e3f-4f76-9833-f22c2e6c56f9" containerName="main" containerID="cri-o://57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596" gracePeriod=45 Jul 23 18:43:57.529682 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.529645 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:57.576440 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.576366 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:43:57.600693 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.600654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:57.675656 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.675616 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tmp-dir\") pod \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " Jul 23 18:43:57.675866 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.675740 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbzwg\" (UniqueName: \"kubernetes.io/projected/893249a5-2e3f-4f76-9833-f22c2e6c56f9-kube-api-access-hbzwg\") pod \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " Jul 23 18:43:57.675866 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.675800 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-home\") pod \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " Jul 23 18:43:57.675866 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.675849 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-model-cache\") pod \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " Jul 23 18:43:57.676067 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.675898 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tls-certs\") pod \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " Jul 23 18:43:57.676067 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.675914 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "893249a5-2e3f-4f76-9833-f22c2e6c56f9" (UID: "893249a5-2e3f-4f76-9833-f22c2e6c56f9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:43:57.676067 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.675924 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-dshm\") pod \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\" (UID: \"893249a5-2e3f-4f76-9833-f22c2e6c56f9\") " Jul 23 18:43:57.676251 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.676073 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-home" (OuterVolumeSpecName: "home") pod "893249a5-2e3f-4f76-9833-f22c2e6c56f9" (UID: "893249a5-2e3f-4f76-9833-f22c2e6c56f9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:43:57.676318 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.676263 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-model-cache" (OuterVolumeSpecName: "model-cache") pod "893249a5-2e3f-4f76-9833-f22c2e6c56f9" (UID: "893249a5-2e3f-4f76-9833-f22c2e6c56f9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:43:57.676399 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.676379 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-home\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:57.676456 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.676401 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-model-cache\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:57.676456 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.676416 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tmp-dir\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:57.678232 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.678193 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-dshm" (OuterVolumeSpecName: "dshm") pod "893249a5-2e3f-4f76-9833-f22c2e6c56f9" (UID: "893249a5-2e3f-4f76-9833-f22c2e6c56f9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:43:57.678370 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.678259 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/893249a5-2e3f-4f76-9833-f22c2e6c56f9-kube-api-access-hbzwg" (OuterVolumeSpecName: "kube-api-access-hbzwg") pod "893249a5-2e3f-4f76-9833-f22c2e6c56f9" (UID: "893249a5-2e3f-4f76-9833-f22c2e6c56f9"). InnerVolumeSpecName "kube-api-access-hbzwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:43:57.678370 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.678328 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "893249a5-2e3f-4f76-9833-f22c2e6c56f9" (UID: "893249a5-2e3f-4f76-9833-f22c2e6c56f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:43:57.777501 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.777466 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/893249a5-2e3f-4f76-9833-f22c2e6c56f9-tls-certs\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:57.777501 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.777497 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/893249a5-2e3f-4f76-9833-f22c2e6c56f9-dshm\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:57.777501 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:57.777506 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbzwg\" (UniqueName: \"kubernetes.io/projected/893249a5-2e3f-4f76-9833-f22c2e6c56f9-kube-api-access-hbzwg\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:58.103121 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.103080 2562 generic.go:356] "Generic (PLEG): container finished" podID="893249a5-2e3f-4f76-9833-f22c2e6c56f9" containerID="57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596" exitCode=0 Jul 23 18:43:58.103391 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.103132 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" Jul 23 18:43:58.103391 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.103169 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" event={"ID":"893249a5-2e3f-4f76-9833-f22c2e6c56f9","Type":"ContainerDied","Data":"57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596"} Jul 23 18:43:58.103391 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.103243 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn" event={"ID":"893249a5-2e3f-4f76-9833-f22c2e6c56f9","Type":"ContainerDied","Data":"04fd028cb0fba8293169f94d9e566bb7a0a6a087de6d03d86256f49a621235d9"} Jul 23 18:43:58.103391 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.103269 2562 scope.go:117] "RemoveContainer" containerID="57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596" Jul 23 18:43:58.113349 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.113327 2562 scope.go:117] "RemoveContainer" containerID="57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596" Jul 23 18:43:58.113689 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:43:58.113670 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596\": container with ID starting with 57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596 not found: ID does not exist" containerID="57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596" Jul 23 18:43:58.113745 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.113700 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596"} err="failed to get container status \"57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596\": rpc error: code = NotFound desc = could not find container \"57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596\": container with ID starting with 57f9a8f231727bc06fa77bbd65030bc6e2dedd6b6eb5ae003456ec9a0eb21596 not found: ID does not exist" Jul 23 18:43:58.125579 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.125548 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn"] Jul 23 18:43:58.129799 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.129771 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/auth-test-model-5f06b1-kserve-6cdd95cf59-g4dkn"] Jul 23 18:43:58.312356 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.312312 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" podUID="40440bd9-635c-43ec-b3cc-77335a574826" containerName="main" containerID="cri-o://34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a" gracePeriod=45 Jul 23 18:43:58.583010 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.582976 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:43:58.597925 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.597889 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:58.684892 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.684789 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-tmp-dir\") pod \"40440bd9-635c-43ec-b3cc-77335a574826\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " Jul 23 18:43:58.684892 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.684867 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40440bd9-635c-43ec-b3cc-77335a574826-tls-certs\") pod \"40440bd9-635c-43ec-b3cc-77335a574826\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " Jul 23 18:43:58.684892 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.684888 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-model-cache\") pod \"40440bd9-635c-43ec-b3cc-77335a574826\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " Jul 23 18:43:58.685197 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.684907 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcqnk\" (UniqueName: \"kubernetes.io/projected/40440bd9-635c-43ec-b3cc-77335a574826-kube-api-access-gcqnk\") pod \"40440bd9-635c-43ec-b3cc-77335a574826\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " Jul 23 18:43:58.685197 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.684950 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-dshm\") pod \"40440bd9-635c-43ec-b3cc-77335a574826\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " Jul 23 18:43:58.685197 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.684996 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-home\") pod \"40440bd9-635c-43ec-b3cc-77335a574826\" (UID: \"40440bd9-635c-43ec-b3cc-77335a574826\") " Jul 23 18:43:58.685197 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.685144 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "40440bd9-635c-43ec-b3cc-77335a574826" (UID: "40440bd9-635c-43ec-b3cc-77335a574826"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:43:58.685420 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.685204 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-model-cache" (OuterVolumeSpecName: "model-cache") pod "40440bd9-635c-43ec-b3cc-77335a574826" (UID: "40440bd9-635c-43ec-b3cc-77335a574826"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:43:58.685420 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.685323 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-home" (OuterVolumeSpecName: "home") pod "40440bd9-635c-43ec-b3cc-77335a574826" (UID: "40440bd9-635c-43ec-b3cc-77335a574826"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:43:58.685495 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.685466 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-tmp-dir\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:58.685495 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.685486 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-model-cache\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:58.685559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.685500 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-home\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:58.687438 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.687410 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-dshm" (OuterVolumeSpecName: "dshm") pod "40440bd9-635c-43ec-b3cc-77335a574826" (UID: "40440bd9-635c-43ec-b3cc-77335a574826"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:43:58.687573 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.687465 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40440bd9-635c-43ec-b3cc-77335a574826-kube-api-access-gcqnk" (OuterVolumeSpecName: "kube-api-access-gcqnk") pod "40440bd9-635c-43ec-b3cc-77335a574826" (UID: "40440bd9-635c-43ec-b3cc-77335a574826"). InnerVolumeSpecName "kube-api-access-gcqnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:43:58.687573 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.687504 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40440bd9-635c-43ec-b3cc-77335a574826-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "40440bd9-635c-43ec-b3cc-77335a574826" (UID: "40440bd9-635c-43ec-b3cc-77335a574826"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:43:58.699533 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.699492 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:58.786464 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.786426 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40440bd9-635c-43ec-b3cc-77335a574826-tls-certs\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:58.786464 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.786458 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcqnk\" (UniqueName: \"kubernetes.io/projected/40440bd9-635c-43ec-b3cc-77335a574826-kube-api-access-gcqnk\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:58.786464 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:58.786469 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40440bd9-635c-43ec-b3cc-77335a574826-dshm\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:43:59.110716 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.110675 2562 generic.go:356] "Generic (PLEG): container finished" podID="40440bd9-635c-43ec-b3cc-77335a574826" containerID="34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a" exitCode=0 Jul 23 18:43:59.110981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.110725 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" Jul 23 18:43:59.110981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.110751 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" event={"ID":"40440bd9-635c-43ec-b3cc-77335a574826","Type":"ContainerDied","Data":"34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a"} Jul 23 18:43:59.110981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.110785 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86" event={"ID":"40440bd9-635c-43ec-b3cc-77335a574826","Type":"ContainerDied","Data":"b96b17ee06b3fbd5a8472db08c85bc69aeebe2016e8006648a33aff4fcd67da7"} Jul 23 18:43:59.110981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.110803 2562 scope.go:117] "RemoveContainer" containerID="34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a" Jul 23 18:43:59.120375 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.120353 2562 scope.go:117] "RemoveContainer" containerID="34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a" Jul 23 18:43:59.120676 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:43:59.120657 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a\": container with ID starting with 34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a not found: ID does not exist" containerID="34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a" Jul 23 18:43:59.120722 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.120688 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a"} err="failed to get container status \"34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a\": rpc error: code = NotFound desc = could not find container \"34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a\": container with ID starting with 34754880c2d618fa42616cfc8f7cb11498ad9bf7a3ed83172a178c176edb377a not found: ID does not exist" Jul 23 18:43:59.130921 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.130880 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="893249a5-2e3f-4f76-9833-f22c2e6c56f9" path="/var/lib/kubelet/pods/893249a5-2e3f-4f76-9833-f22c2e6c56f9/volumes" Jul 23 18:43:59.133021 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.132997 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86"] Jul 23 18:43:59.137840 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.137814 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/auth-test-model-8ee985-kserve-59bbbb8f9-tlp86"] Jul 23 18:43:59.534628 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.534584 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:59.633187 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:43:59.633146 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:00.512244 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:00.512177 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:00.613308 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:00.613272 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:01.131247 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:01.131195 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40440bd9-635c-43ec-b3cc-77335a574826" path="/var/lib/kubelet/pods/40440bd9-635c-43ec-b3cc-77335a574826/volumes" Jul 23 18:44:01.328155 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:01.328116 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:01.424105 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:01.423999 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:02.190726 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:02.190689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:02.291854 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:02.291815 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:03.012680 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:03.012640 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:03.112588 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:03.112551 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:03.971265 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:03.971203 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:04.071057 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:04.071012 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:04.772224 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:04.772170 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:04.876958 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:04.876915 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:05.721268 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:05.721201 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:05.821133 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:05.821096 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:06.628308 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:06.628272 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:06.721779 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:06.721738 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:07.462073 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:07.462034 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:07.563780 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:07.563737 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:08.282767 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:08.282718 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:08.388084 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:08.388039 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:09.239202 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.239128 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:09.339443 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.339407 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:09.675275 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.675157 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m"] Jul 23 18:44:09.675669 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.675650 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40440bd9-635c-43ec-b3cc-77335a574826" containerName="main" Jul 23 18:44:09.675767 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.675671 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="40440bd9-635c-43ec-b3cc-77335a574826" containerName="main" Jul 23 18:44:09.675767 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.675730 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="893249a5-2e3f-4f76-9833-f22c2e6c56f9" containerName="main" Jul 23 18:44:09.675767 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.675738 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="893249a5-2e3f-4f76-9833-f22c2e6c56f9" containerName="main" Jul 23 18:44:09.675892 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.675824 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="893249a5-2e3f-4f76-9833-f22c2e6c56f9" containerName="main" Jul 23 18:44:09.675892 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.675837 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="40440bd9-635c-43ec-b3cc-77335a574826" containerName="main" Jul 23 18:44:09.680618 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.680591 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.683177 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.683153 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"default-dockercfg-zfz4b\"" Jul 23 18:44:09.684163 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.684120 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"sub-test-model-5f06b1-kserve-self-signed-certs\"" Jul 23 18:44:09.684163 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.684142 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"kube-root-ca.crt\"" Jul 23 18:44:09.684376 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.684290 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"openshift-service-ca.crt\"" Jul 23 18:44:09.695059 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.695034 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m"] Jul 23 18:44:09.713455 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.713401 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:09.788453 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.788407 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-dshm\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.788453 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.788453 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tmp-dir\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.788742 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.788485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-model-cache\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.788742 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.788508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tls-certs\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.788742 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.788537 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-home\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.788742 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.788627 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbb2z\" (UniqueName: \"kubernetes.io/projected/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-kube-api-access-mbb2z\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.889592 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.889542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-dshm\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.889592 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.889592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tmp-dir\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.889887 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.889627 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-model-cache\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.889887 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.889653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tls-certs\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.889887 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.889683 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-home\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.889887 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.889769 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbb2z\" (UniqueName: \"kubernetes.io/projected/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-kube-api-access-mbb2z\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.890131 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.890104 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-model-cache\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.890200 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.890121 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tmp-dir\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.890200 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.890183 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-home\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.892095 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.892072 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-dshm\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.892380 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.892362 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tls-certs\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.898597 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.898570 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbb2z\" (UniqueName: \"kubernetes.io/projected/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-kube-api-access-mbb2z\") pod \"sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:09.993746 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:09.993701 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:10.035611 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:10.035571 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:10.135531 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:10.135491 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:10.142535 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:10.142412 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m"] Jul 23 18:44:10.148474 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:44:10.148435 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2367fde_4059_4eaf_a28c_aeba4f0c9dbc.slice/crio-313b73804b5ccd8568afbc985def2fe246db36378b4256efaa9ace1201b6e347 WatchSource:0}: Error finding container 313b73804b5ccd8568afbc985def2fe246db36378b4256efaa9ace1201b6e347: Status 404 returned error can't find the container with id 313b73804b5ccd8568afbc985def2fe246db36378b4256efaa9ace1201b6e347 Jul 23 18:44:10.165268 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:10.165239 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" event={"ID":"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc","Type":"ContainerStarted","Data":"313b73804b5ccd8568afbc985def2fe246db36378b4256efaa9ace1201b6e347"} Jul 23 18:44:10.937650 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:10.937612 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:11.038178 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:11.038136 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:11.172893 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:11.172858 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" event={"ID":"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc","Type":"ContainerStarted","Data":"798907fdd118d8ca8162c70b554052e252ab69e4785dc70114ddd99337d79837"} Jul 23 18:44:11.191850 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:11.191733 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" podStartSLOduration=2.191710396 podStartE2EDuration="2.191710396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:44:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:44:11.18995609 +0000 UTC m=+2672.677448083" watchObservedRunningTime="2026-07-23 18:44:11.191710396 +0000 UTC m=+2672.679202388" Jul 23 18:44:11.813231 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:11.813161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:11.914487 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:11.914444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:12.698497 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:12.698448 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:12.800189 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:12.800148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:13.066518 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:13.066423 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:44:13.862887 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:13.862832 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:13.963083 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:13.963046 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:14.661928 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:14.661880 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:14.761929 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:14.761892 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:15.487891 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:15.487855 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:15.572258 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:15.572193 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:16.514093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:16.514055 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:16.614019 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:16.613985 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:17.283510 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:17.283472 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:17.381337 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:17.381299 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:18.164199 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:18.164161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:18.264360 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:18.264323 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:19.307928 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:19.307896 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:19.409756 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:19.409709 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:19.994026 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:19.993981 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:19.994256 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:19.994059 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:20.006868 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:20.006839 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:20.074656 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:20.074607 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:20.174456 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:20.174424 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:20.229857 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:20.229826 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:44:20.842846 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:20.842813 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:20.943156 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:20.943109 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:21.787402 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:21.787358 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:21.887418 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:21.887386 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:22.542406 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:22.542371 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:22.643906 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:22.643870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:23.311471 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:23.311436 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:23.426121 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:23.426078 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:24.349636 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:24.349587 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:24.450044 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:24.450008 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:25.114465 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:25.114425 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:25.214315 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:25.214280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:26.020526 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:26.020489 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:26.121628 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:26.121583 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:26.804143 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:26.804103 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:26.904669 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:26.904618 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:27.145694 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:27.145605 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:27.699061 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:27.699026 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:27.800425 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:27.800386 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:28.301335 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:28.301296 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:28.401887 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:28.401848 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:28.502470 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:28.502432 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:28.640376 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:28.640280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:29.563224 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:29.563174 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:29.663400 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:29.663367 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:30.426021 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:30.425980 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:30.525956 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:30.525918 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:31.323959 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:31.323914 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:31.424184 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:31.424143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:32.111818 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:32.111768 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:32.212712 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:32.212673 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:33.030445 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:33.030402 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:33.130627 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:33.130583 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:33.956158 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:33.956120 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:34.052772 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:34.052734 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:34.989797 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:34.989766 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:35.091946 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:35.091910 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:35.788720 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:35.788680 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:35.887624 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:35.887582 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:36.695258 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:36.695199 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:36.795464 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:36.795423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:37.514625 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:37.514583 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:37.611087 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:37.611040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:38.403176 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:38.403126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:38.504029 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:38.503991 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:39.174901 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:39.174854 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:39.189722 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:39.189692 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:44:39.196642 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:39.196614 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:44:39.275625 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:39.275558 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:40.198676 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:40.198635 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:40.301765 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:40.301725 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:41.101012 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:41.100976 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:41.193335 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:41.193297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:41.899300 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:41.899266 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:42.012470 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:42.012433 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:42.725657 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:42.725619 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:43.262646 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:43.262610 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:43.770965 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:43.770926 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:43.872630 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:43.872587 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:44.666850 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:44.666809 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:44.767116 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:44.767087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:45.628528 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:45.628479 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:45.727100 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:45.727064 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:46.453626 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:46.453591 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:46.549579 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:46.549542 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:47.480695 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:47.480644 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:47.578541 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:47.578503 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:47.821062 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:47.820968 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:48.355533 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.355497 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:48.365969 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.365936 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt"] Jul 23 18:44:48.369319 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.369294 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.372070 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.372041 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"kube-root-ca.crt\"" Jul 23 18:44:48.373301 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.373279 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"default-dockercfg-x4v54\"" Jul 23 18:44:48.373301 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.373289 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"sub-test-model-8ee985-kserve-self-signed-certs\"" Jul 23 18:44:48.373498 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.373293 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"openshift-service-ca.crt\"" Jul 23 18:44:48.379852 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.379828 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt"] Jul 23 18:44:48.433406 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.433371 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-tmp-dir\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.433406 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.433409 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/779adfee-9c83-4af0-a3c9-04555d6c0366-tls-certs\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.433624 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.433469 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5t6z\" (UniqueName: \"kubernetes.io/projected/779adfee-9c83-4af0-a3c9-04555d6c0366-kube-api-access-j5t6z\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.433624 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.433569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-dshm\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.433624 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.433592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-home\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.433624 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.433607 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-model-cache\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.454951 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.454916 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:48.534655 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.534609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-dshm\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.534655 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.534656 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-home\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.535176 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.534687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-model-cache\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.535176 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.534797 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-tmp-dir\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.535176 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.534845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/779adfee-9c83-4af0-a3c9-04555d6c0366-tls-certs\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.535176 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.534904 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5t6z\" (UniqueName: \"kubernetes.io/projected/779adfee-9c83-4af0-a3c9-04555d6c0366-kube-api-access-j5t6z\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.535176 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.535141 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-model-cache\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.535507 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.535325 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-home\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.535507 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.535382 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-tmp-dir\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.537076 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.537056 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-dshm\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.537411 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.537395 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/779adfee-9c83-4af0-a3c9-04555d6c0366-tls-certs\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.547106 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.547069 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5t6z\" (UniqueName: \"kubernetes.io/projected/779adfee-9c83-4af0-a3c9-04555d6c0366-kube-api-access-j5t6z\") pod \"sub-test-model-8ee985-kserve-655f79c995-45rkt\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.680841 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.680747 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:48.832025 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:48.831989 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt"] Jul 23 18:44:48.835304 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:44:48.835273 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod779adfee_9c83_4af0_a3c9_04555d6c0366.slice/crio-4b50a5bf36e59d7dbbec0563ab2091c2ed870db4c043dd02c33f65c4f4f590f4 WatchSource:0}: Error finding container 4b50a5bf36e59d7dbbec0563ab2091c2ed870db4c043dd02c33f65c4f4f590f4: Status 404 returned error can't find the container with id 4b50a5bf36e59d7dbbec0563ab2091c2ed870db4c043dd02c33f65c4f4f590f4 Jul 23 18:44:49.363933 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:49.363894 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" event={"ID":"779adfee-9c83-4af0-a3c9-04555d6c0366","Type":"ContainerStarted","Data":"0c4be940f78703bb540e05ed5fcbc632920f66c552c8dae3408b0425a6117aab"} Jul 23 18:44:49.363933 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:49.363940 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" event={"ID":"779adfee-9c83-4af0-a3c9-04555d6c0366","Type":"ContainerStarted","Data":"4b50a5bf36e59d7dbbec0563ab2091c2ed870db4c043dd02c33f65c4f4f590f4"} Jul 23 18:44:49.385661 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:49.385589 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" podStartSLOduration=1.3855663599999999 podStartE2EDuration="1.38556636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:44:49.381671865 +0000 UTC m=+2710.869163857" watchObservedRunningTime="2026-07-23 18:44:49.38556636 +0000 UTC m=+2710.873058351" Jul 23 18:44:49.417455 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:49.417418 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:49.516705 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:49.516665 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:50.145908 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:50.145867 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:50.248328 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:50.248297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:51.154048 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:51.154014 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:51.254912 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:51.254879 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:51.962168 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:51.962134 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:44:52.066307 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:52.066270 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:52.166979 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:52.166943 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:53.024235 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:53.024167 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:53.123864 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:53.123829 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:53.815839 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:53.815807 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:53.915832 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:53.915795 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:54.797427 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:54.797393 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:55.398343 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:55.398305 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:55.623187 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:55.623148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:55.724329 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:55.724298 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:56.510454 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:56.510423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:56.611288 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:56.611248 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:57.267485 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:57.267446 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:57.369269 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:57.369235 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:58.241438 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:58.241404 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:58.348281 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:58.348245 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:58.681276 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:58.681140 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:58.681276 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:58.681205 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:58.694047 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:58.694017 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:44:59.212712 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:59.212662 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:59.310702 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:59.310663 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:59.422509 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:44:59.422475 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:45:00.168343 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:00.168308 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:00.266377 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:00.266332 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:01.205347 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:01.205313 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:01.309693 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:01.309661 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:02.263041 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:02.263000 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:02.359720 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:02.359685 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:03.324827 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:03.324788 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:03.448995 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:03.448957 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:04.286674 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:04.286637 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:04.388283 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:04.388239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:05.212005 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:05.211971 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:05.310896 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:05.310859 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:06.266165 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:06.266125 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:06.367845 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:06.367811 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:07.354604 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:07.354572 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:07.455523 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:07.455486 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:07.776569 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:07.776534 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:45:08.479649 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:08.479614 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:08.580006 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:08.579967 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:09.413887 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:09.413852 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:09.520529 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:09.520498 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:10.497102 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:10.497048 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:10.594914 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:10.594873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:11.537362 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:11.537320 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:11.637420 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:11.637382 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:12.558144 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:12.558107 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:12.658992 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:12.658956 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:13.669423 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:13.669389 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:13.784179 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:13.784145 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:14.967367 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:14.967329 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:15.065438 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:15.065399 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:15.809574 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:15.809538 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:15.909069 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:15.909034 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:16.751118 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:16.751084 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:16.851669 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:16.851632 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:17.907735 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:17.907696 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:18.010743 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:18.010705 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:18.879898 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:18.879861 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:18.972976 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:18.972931 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:20.073425 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:20.073385 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:20.168584 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:20.168544 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:21.182513 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:21.182464 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:21.323871 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:21.323824 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:21.621518 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:21.621481 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:21.734536 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:21.734496 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:22.204548 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:22.204512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:22.305362 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:22.305327 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:23.122704 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:23.122670 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:23.220114 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:23.220075 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:24.129320 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:24.129291 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:24.230185 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:24.230136 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:24.985265 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:24.985232 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:25.085297 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:25.085261 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:25.927721 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:25.927689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:26.026406 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:26.026355 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:26.750147 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:26.750113 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:26.851847 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:26.851811 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:27.750023 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:27.749983 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:27.866192 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:27.866152 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:28.262428 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:28.262383 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:45:28.624245 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:28.624125 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:28.723279 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:28.723241 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:29.720608 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:29.720571 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:29.826586 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:29.826541 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:30.636504 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:30.636471 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:30.736951 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:30.736906 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:31.536387 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:31.536344 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:31.634807 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:31.634772 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:32.626009 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:32.625961 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:32.723482 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:32.723444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:33.416114 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:33.416070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:33.516863 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:33.516826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:34.367805 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:34.367759 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:34.485290 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:34.485253 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m"] Jul 23 18:45:34.584817 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:34.584779 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:34.686572 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:34.686470 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:35.397637 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:35.397599 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:35.457155 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:35.457120 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt"] Jul 23 18:45:35.492275 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:35.492238 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:35.578443 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:35.578387 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:36.431604 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:36.431561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:36.523749 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:36.523707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:37.410434 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:37.410363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:37.559457 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:37.559415 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:38.751513 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:38.751473 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:38.855724 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:38.855685 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:39.845678 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:39.845636 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:39.950069 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:39.950027 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:41.132894 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:41.132864 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:41.277449 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:41.277409 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:42.543545 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:42.543502 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:42.778980 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:42.778933 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:43.922332 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:43.922286 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:44.058199 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:44.058166 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:45.149182 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:45.149139 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:45.275005 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:45.274972 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:46.430163 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:46.430125 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:46.550506 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:46.550462 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:47.142437 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:47.142400 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:45:47.787556 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:47.787509 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:47.904523 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:47.904474 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:48.846267 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:48.846200 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:48.956134 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:48.956093 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:49.506810 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.506744 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" podUID="c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" containerName="main" containerID="cri-o://798907fdd118d8ca8162c70b554052e252ab69e4785dc70114ddd99337d79837" gracePeriod=45 Jul 23 18:45:49.672446 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.672413 2562 generic.go:356] "Generic (PLEG): container finished" podID="c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" containerID="798907fdd118d8ca8162c70b554052e252ab69e4785dc70114ddd99337d79837" exitCode=0 Jul 23 18:45:49.672601 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.672498 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" event={"ID":"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc","Type":"ContainerDied","Data":"798907fdd118d8ca8162c70b554052e252ab69e4785dc70114ddd99337d79837"} Jul 23 18:45:49.772423 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.772350 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:45:49.823253 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823197 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tmp-dir\") pod \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " Jul 23 18:45:49.823429 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823278 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-model-cache\") pod \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " Jul 23 18:45:49.823429 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823345 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbb2z\" (UniqueName: \"kubernetes.io/projected/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-kube-api-access-mbb2z\") pod \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " Jul 23 18:45:49.823429 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823382 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-home\") pod \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " Jul 23 18:45:49.823596 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823431 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tls-certs\") pod \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " Jul 23 18:45:49.823596 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823466 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-dshm\") pod \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\" (UID: \"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc\") " Jul 23 18:45:49.823596 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823498 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" (UID: "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:49.823596 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823586 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-model-cache" (OuterVolumeSpecName: "model-cache") pod "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" (UID: "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:49.823808 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823681 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-home" (OuterVolumeSpecName: "home") pod "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" (UID: "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:49.823862 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823836 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tmp-dir\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:49.823862 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823859 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-model-cache\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:49.823930 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.823868 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-home\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:49.825663 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.825635 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-kube-api-access-mbb2z" (OuterVolumeSpecName: "kube-api-access-mbb2z") pod "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" (UID: "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc"). InnerVolumeSpecName "kube-api-access-mbb2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:45:49.825663 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.825653 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-dshm" (OuterVolumeSpecName: "dshm") pod "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" (UID: "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:49.825663 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.825656 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" (UID: "c2367fde-4059-4eaf-a28c-aeba4f0c9dbc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:45:49.924863 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.924821 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-tls-certs\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:49.924863 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.924863 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-dshm\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:49.925355 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:49.924875 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbb2z\" (UniqueName: \"kubernetes.io/projected/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc-kube-api-access-mbb2z\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:50.005979 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.005930 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:50.108408 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.108315 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:50.477517 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.477469 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" podUID="779adfee-9c83-4af0-a3c9-04555d6c0366" containerName="main" containerID="cri-o://0c4be940f78703bb540e05ed5fcbc632920f66c552c8dae3408b0425a6117aab" gracePeriod=45 Jul 23 18:45:50.680844 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.680806 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" Jul 23 18:45:50.681055 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.680811 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m" event={"ID":"c2367fde-4059-4eaf-a28c-aeba4f0c9dbc","Type":"ContainerDied","Data":"313b73804b5ccd8568afbc985def2fe246db36378b4256efaa9ace1201b6e347"} Jul 23 18:45:50.681055 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.680964 2562 scope.go:117] "RemoveContainer" containerID="798907fdd118d8ca8162c70b554052e252ab69e4785dc70114ddd99337d79837" Jul 23 18:45:50.683113 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.683081 2562 generic.go:356] "Generic (PLEG): container finished" podID="779adfee-9c83-4af0-a3c9-04555d6c0366" containerID="0c4be940f78703bb540e05ed5fcbc632920f66c552c8dae3408b0425a6117aab" exitCode=0 Jul 23 18:45:50.683268 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.683146 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" event={"ID":"779adfee-9c83-4af0-a3c9-04555d6c0366","Type":"ContainerDied","Data":"0c4be940f78703bb540e05ed5fcbc632920f66c552c8dae3408b0425a6117aab"} Jul 23 18:45:50.711270 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.711234 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m"] Jul 23 18:45:50.717543 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.717519 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/sub-test-model-5f06b1-kserve-666bf9b9c9-f7d6m"] Jul 23 18:45:50.740074 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.739999 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:45:50.832065 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.832029 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-dshm\") pod \"779adfee-9c83-4af0-a3c9-04555d6c0366\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " Jul 23 18:45:50.832065 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.832067 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-tmp-dir\") pod \"779adfee-9c83-4af0-a3c9-04555d6c0366\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " Jul 23 18:45:50.832350 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.832125 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-model-cache\") pod \"779adfee-9c83-4af0-a3c9-04555d6c0366\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " Jul 23 18:45:50.832350 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.832152 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5t6z\" (UniqueName: \"kubernetes.io/projected/779adfee-9c83-4af0-a3c9-04555d6c0366-kube-api-access-j5t6z\") pod \"779adfee-9c83-4af0-a3c9-04555d6c0366\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " Jul 23 18:45:50.832350 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.832184 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/779adfee-9c83-4af0-a3c9-04555d6c0366-tls-certs\") pod \"779adfee-9c83-4af0-a3c9-04555d6c0366\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " Jul 23 18:45:50.832350 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.832223 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-home\") pod \"779adfee-9c83-4af0-a3c9-04555d6c0366\" (UID: \"779adfee-9c83-4af0-a3c9-04555d6c0366\") " Jul 23 18:45:50.832554 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.832439 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-model-cache" (OuterVolumeSpecName: "model-cache") pod "779adfee-9c83-4af0-a3c9-04555d6c0366" (UID: "779adfee-9c83-4af0-a3c9-04555d6c0366"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:50.832554 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.832449 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "779adfee-9c83-4af0-a3c9-04555d6c0366" (UID: "779adfee-9c83-4af0-a3c9-04555d6c0366"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:50.832665 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.832592 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-tmp-dir\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:50.832665 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.832611 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-model-cache\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:50.832770 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.832715 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-home" (OuterVolumeSpecName: "home") pod "779adfee-9c83-4af0-a3c9-04555d6c0366" (UID: "779adfee-9c83-4af0-a3c9-04555d6c0366"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:50.834365 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.834339 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/779adfee-9c83-4af0-a3c9-04555d6c0366-kube-api-access-j5t6z" (OuterVolumeSpecName: "kube-api-access-j5t6z") pod "779adfee-9c83-4af0-a3c9-04555d6c0366" (UID: "779adfee-9c83-4af0-a3c9-04555d6c0366"). InnerVolumeSpecName "kube-api-access-j5t6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:45:50.834666 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.834651 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-dshm" (OuterVolumeSpecName: "dshm") pod "779adfee-9c83-4af0-a3c9-04555d6c0366" (UID: "779adfee-9c83-4af0-a3c9-04555d6c0366"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:50.834744 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.834732 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/779adfee-9c83-4af0-a3c9-04555d6c0366-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "779adfee-9c83-4af0-a3c9-04555d6c0366" (UID: "779adfee-9c83-4af0-a3c9-04555d6c0366"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:45:50.933512 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.933469 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-dshm\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:50.933512 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.933503 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5t6z\" (UniqueName: \"kubernetes.io/projected/779adfee-9c83-4af0-a3c9-04555d6c0366-kube-api-access-j5t6z\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:50.933512 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.933523 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/779adfee-9c83-4af0-a3c9-04555d6c0366-tls-certs\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:50.933992 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.933535 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/779adfee-9c83-4af0-a3c9-04555d6c0366-home\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:45:50.992389 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:50.992288 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:51.091814 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:51.091764 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:51.131054 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:51.131018 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" path="/var/lib/kubelet/pods/c2367fde-4059-4eaf-a28c-aeba4f0c9dbc/volumes" Jul 23 18:45:51.696847 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:51.696812 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" Jul 23 18:45:51.697065 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:51.696839 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt" event={"ID":"779adfee-9c83-4af0-a3c9-04555d6c0366","Type":"ContainerDied","Data":"4b50a5bf36e59d7dbbec0563ab2091c2ed870db4c043dd02c33f65c4f4f590f4"} Jul 23 18:45:51.697065 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:51.696901 2562 scope.go:117] "RemoveContainer" containerID="0c4be940f78703bb540e05ed5fcbc632920f66c552c8dae3408b0425a6117aab" Jul 23 18:45:51.718312 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:51.718285 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt"] Jul 23 18:45:51.727475 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:51.727448 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/sub-test-model-8ee985-kserve-655f79c995-45rkt"] Jul 23 18:45:52.148255 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:52.148148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:52.248725 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:52.248685 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:53.130837 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:53.130800 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="779adfee-9c83-4af0-a3c9-04555d6c0366" path="/var/lib/kubelet/pods/779adfee-9c83-4af0-a3c9-04555d6c0366/volumes" Jul 23 18:45:53.263723 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:53.263688 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:53.368295 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:53.368242 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:54.271009 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:54.270967 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:54.373434 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:54.373395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:55.119136 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:55.119102 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:55.225344 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:55.225311 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:56.096448 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:56.096412 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:56.194088 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:56.194052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:56.948194 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:56.948156 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:57.120647 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:57.120606 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:57.847181 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:57.847135 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:57.947990 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:57.947944 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:58.770772 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:58.770728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:58.872092 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:58.872055 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:59.804733 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:59.804694 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:59.905552 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:45:59.905517 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:00.763471 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:00.763380 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:00.866156 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:00.866119 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:01.761236 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:01.761179 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:01.858659 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:01.858611 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:02.581429 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:02.581395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:02.682925 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:02.682890 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:03.462703 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:03.462670 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:03.595329 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:03.595296 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:04.806389 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:04.806349 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:04.908456 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:04.908424 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:05.770547 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:05.770507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:05.870176 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:05.870139 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:06.677206 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:06.677171 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:06.780746 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:06.780715 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:07.932383 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:07.932328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:08.031038 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:08.031003 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:08.816266 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:08.816226 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:08.913542 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:08.913491 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:09.681376 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:09.681335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:09.787595 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:09.787535 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:10.837450 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:10.837413 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:10.939089 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:10.939040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:11.698005 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:11.697963 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:11.798089 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:11.798052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:12.595298 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:12.595263 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:12.697438 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:12.697401 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:13.479519 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:13.479482 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:13.577637 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:13.577598 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:14.571355 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:14.571319 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:14.671705 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:14.671671 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:15.205565 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:15.205525 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:15.305968 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:15.305931 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:16.139519 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:16.139484 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:16.397795 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:16.397683 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:17.042747 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:17.042707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:17.139938 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:17.139891 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:17.882243 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:17.882183 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:17.982558 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:17.982522 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:18.742510 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:18.742479 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:18.840438 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:18.840408 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:19.645022 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:19.644987 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:19.742614 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:19.742574 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:20.524940 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:20.524900 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:20.624500 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:20.624460 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:21.427373 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:21.427334 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:21.528397 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:21.528360 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:22.255202 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.255166 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:22.358441 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.358406 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:22.566069 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.565956 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd"] Jul 23 18:46:22.566595 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.566549 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" containerName="main" Jul 23 18:46:22.566595 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.566572 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" containerName="main" Jul 23 18:46:22.566595 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.566585 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="779adfee-9c83-4af0-a3c9-04555d6c0366" containerName="main" Jul 23 18:46:22.566595 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.566593 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="779adfee-9c83-4af0-a3c9-04555d6c0366" containerName="main" Jul 23 18:46:22.566818 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.566692 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2367fde-4059-4eaf-a28c-aeba4f0c9dbc" containerName="main" Jul 23 18:46:22.566818 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.566709 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="779adfee-9c83-4af0-a3c9-04555d6c0366" containerName="main" Jul 23 18:46:22.569206 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.569181 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.572800 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.572775 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"openshift-service-ca.crt\"" Jul 23 18:46:22.572951 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.572778 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"default-dockercfg-zfz4b\"" Jul 23 18:46:22.572951 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.572816 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"kube-root-ca.crt\"" Jul 23 18:46:22.572951 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.572848 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f06b1\"/\"rate-test-model-5f06b1-kserve-self-signed-certs\"" Jul 23 18:46:22.580384 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.580359 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd"] Jul 23 18:46:22.669424 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.669367 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:22.719101 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.719064 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-dshm\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.719101 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.719099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-665rd\" (UniqueName: \"kubernetes.io/projected/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-kube-api-access-665rd\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.719396 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.719124 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-home\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.719396 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.719274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-model-cache\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.719396 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.719301 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tls-certs\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.719396 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.719325 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tmp-dir\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.819979 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.819875 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-dshm\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.819979 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.819919 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-665rd\" (UniqueName: \"kubernetes.io/projected/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-kube-api-access-665rd\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.819979 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.819942 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-home\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.820326 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.819993 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-model-cache\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.820326 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.820011 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tls-certs\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.820326 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.820030 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tmp-dir\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.820497 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.820441 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-model-cache\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.820497 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.820482 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-home\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.820582 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.820530 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tmp-dir\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.822337 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.822313 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-dshm\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.822722 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.822701 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tls-certs\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.829561 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.829530 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-665rd\" (UniqueName: \"kubernetes.io/projected/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-kube-api-access-665rd\") pod \"rate-test-model-5f06b1-kserve-5987fc464c-thczd\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:22.880155 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:22.880109 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:23.034981 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:23.034953 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd"] Jul 23 18:46:23.037570 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:46:23.037540 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9ac055c_fd6d_4a70_97e4_8f260cfbafc9.slice/crio-53a204d12061ac229ba410e72ef39e7963dcdba7ebe718e83fd550b3e492854f WatchSource:0}: Error finding container 53a204d12061ac229ba410e72ef39e7963dcdba7ebe718e83fd550b3e492854f: Status 404 returned error can't find the container with id 53a204d12061ac229ba410e72ef39e7963dcdba7ebe718e83fd550b3e492854f Jul 23 18:46:23.082263 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:23.082224 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:23.182661 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:23.182614 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:23.862618 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:23.862573 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" event={"ID":"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9","Type":"ContainerStarted","Data":"9487d1bcf03854204e3ae88fadbedd66ae8a9310dd9ba4733cee47e4a3362009"} Jul 23 18:46:23.863041 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:23.862630 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" event={"ID":"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9","Type":"ContainerStarted","Data":"53a204d12061ac229ba410e72ef39e7963dcdba7ebe718e83fd550b3e492854f"} Jul 23 18:46:23.891785 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:23.891724 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" podStartSLOduration=1.8917069469999999 podStartE2EDuration="1.891706947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:46:23.889121364 +0000 UTC m=+2805.376613381" watchObservedRunningTime="2026-07-23 18:46:23.891706947 +0000 UTC m=+2805.379198938" Jul 23 18:46:24.224502 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:24.224462 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:24.334529 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:24.334491 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:25.198614 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:25.198574 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:25.299615 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:25.299583 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:26.050920 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:26.050889 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:26.151119 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:26.151079 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:26.365941 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:26.365838 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:46:27.055633 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:27.055596 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:27.155920 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:27.155880 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:28.088721 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:28.088633 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:28.187802 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:28.187767 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:29.276301 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:29.276264 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:29.363131 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:29.363090 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:30.177557 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:30.177518 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:30.281698 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:30.281664 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:31.163128 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:31.163088 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:31.263021 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:31.262984 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:31.946281 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:31.946200 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:32.048997 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:32.048957 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:32.880645 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:32.880608 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:32.880645 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:32.880656 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:32.894079 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:32.894044 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:32.895042 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:32.895012 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:32.918663 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:32.918631 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:46:32.995168 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:32.995132 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:33.796285 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:33.796245 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:33.891902 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:33.891867 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:34.591440 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:34.591404 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:34.689379 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:34.689337 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:35.656471 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:35.656440 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:35.754602 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:35.754559 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:36.531262 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:36.531196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:36.623887 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:36.623848 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:37.587834 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:37.587788 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:37.688818 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:37.688782 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:38.452748 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:38.452716 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:38.553385 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:38.553349 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:39.420053 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:39.419994 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:39.522122 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:39.522093 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:40.279244 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:40.279195 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:41.414121 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:41.414086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:41.516802 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:41.516770 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:42.019727 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:42.019685 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:42.119773 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:42.119741 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:42.812811 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:42.812777 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:42.913633 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:42.913598 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:43.734631 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:43.734596 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:43.835028 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:43.834989 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:44.627164 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:44.627126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:44.728390 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:44.728354 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:45.422411 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:45.422378 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:45.527025 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:45.526981 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:46.319730 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:46.319683 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:46.421016 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:46.420985 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:48.146280 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:48.146247 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:48.318456 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:48.318419 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:48.430788 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:48.430688 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:49.275842 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.275806 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:49.378639 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.378597 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:49.763189 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.763155 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8"] Jul 23 18:46:49.766662 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.766635 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.770310 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.770280 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"default-dockercfg-x4v54\"" Jul 23 18:46:49.770449 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.770280 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"kube-root-ca.crt\"" Jul 23 18:46:49.770449 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.770289 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"rate-test-model-8ee985-kserve-self-signed-certs\"" Jul 23 18:46:49.770449 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.770314 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ee985\"/\"openshift-service-ca.crt\"" Jul 23 18:46:49.779706 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.779677 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8"] Jul 23 18:46:49.799793 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.799760 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:49.847017 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.846978 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-model-cache\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.847231 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.847031 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-tmp-dir\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.847231 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.847142 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-home\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.847231 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.847165 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-dshm\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.847231 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.847188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49nwk\" (UniqueName: \"kubernetes.io/projected/a8c078ec-1347-42b5-8e1a-c3196806051f-kube-api-access-49nwk\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.847443 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.847280 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c078ec-1347-42b5-8e1a-c3196806051f-tls-certs\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.948309 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.948271 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-model-cache\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.948309 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.948317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-tmp-dir\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.948577 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.948367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-home\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.948577 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.948467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-dshm\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.948577 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.948500 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49nwk\" (UniqueName: \"kubernetes.io/projected/a8c078ec-1347-42b5-8e1a-c3196806051f-kube-api-access-49nwk\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.948577 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.948539 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c078ec-1347-42b5-8e1a-c3196806051f-tls-certs\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.948851 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.948754 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-model-cache\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.948851 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.948827 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-tmp-dir\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.948921 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.948870 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-home\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.950839 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.950816 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-dshm\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.951115 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.951088 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c078ec-1347-42b5-8e1a-c3196806051f-tls-certs\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:49.958028 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:49.958003 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49nwk\" (UniqueName: \"kubernetes.io/projected/a8c078ec-1347-42b5-8e1a-c3196806051f-kube-api-access-49nwk\") pod \"rate-test-model-8ee985-kserve-bb646cdc5-hntg8\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:50.076074 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:50.075971 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:46:50.213689 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:50.213663 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8"] Jul 23 18:46:50.216685 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:46:50.216656 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8c078ec_1347_42b5_8e1a_c3196806051f.slice/crio-2132bdfc15c8d71d75161c61faa3a908ad01b16d63c331f4f53e91bd73bbf861 WatchSource:0}: Error finding container 2132bdfc15c8d71d75161c61faa3a908ad01b16d63c331f4f53e91bd73bbf861: Status 404 returned error can't find the container with id 2132bdfc15c8d71d75161c61faa3a908ad01b16d63c331f4f53e91bd73bbf861 Jul 23 18:46:50.247378 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:50.247348 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:50.346943 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:50.346853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:50.990464 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:50.990422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" event={"ID":"a8c078ec-1347-42b5-8e1a-c3196806051f","Type":"ContainerStarted","Data":"5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498"} Jul 23 18:46:50.990642 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:50.990475 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" event={"ID":"a8c078ec-1347-42b5-8e1a-c3196806051f","Type":"ContainerStarted","Data":"2132bdfc15c8d71d75161c61faa3a908ad01b16d63c331f4f53e91bd73bbf861"} Jul 23 18:46:51.011907 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:51.011856 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" podStartSLOduration=2.011838812 podStartE2EDuration="2.011838812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:46:51.010613006 +0000 UTC m=+2832.498105024" watchObservedRunningTime="2026-07-23 18:46:51.011838812 +0000 UTC m=+2832.499330803" Jul 23 18:46:51.602058 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:51.602018 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:51.703116 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:51.703079 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:52.043744 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:52.043689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:52.144704 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:52.144666 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:52.840500 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:52.840456 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:52.941315 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:52.941280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:53.563322 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:53.563279 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:46:53.614418 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:53.614385 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:53.727884 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:53.727850 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:53.964538 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:53.964501 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:54.513229 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:54.513180 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:54.613673 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:54.613635 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:55.426398 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:55.426366 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:55.527945 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:55.527906 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:56.235542 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:56.235499 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:56.346159 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:56.346121 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:57.195089 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:57.195032 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:57.296616 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:57.296576 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:58.013659 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:58.013615 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:58.113477 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:58.113441 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:58.820560 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:58.820518 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:58.920675 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:58.920637 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:59.885074 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:59.885036 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:59.985515 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:46:59.985443 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:00.076655 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:00.076614 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:47:00.076655 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:00.076670 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:47:00.089967 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:00.089929 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:47:00.715282 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:00.715242 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:00.815865 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:00.815823 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:01.056256 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:01.056127 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:47:01.601317 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:01.601271 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:01.702907 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:01.702871 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:02.554410 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:02.554352 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:02.655599 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:02.655532 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:03.391416 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:03.391377 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:03.491067 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:03.491029 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:04.382033 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:04.381993 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:04.477812 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:04.477777 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:05.516556 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:05.516514 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:05.615561 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:05.615523 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:06.350306 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:06.350269 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:06.451977 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:06.451939 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:07.265272 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:07.265235 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:07.364488 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:07.364454 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:08.389936 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:08.389895 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:08.571396 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:08.571352 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:09.500784 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:09.500744 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:09.621018 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:09.620980 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:10.526767 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:10.526725 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:10.623058 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:10.623001 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:11.483630 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:11.483589 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:11.584756 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:11.584691 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:12.059689 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:12.059652 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:47:12.273527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:12.273481 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:12.374006 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:12.373907 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:13.209607 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:13.209567 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:13.309398 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:13.309359 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:14.040017 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:14.039971 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:14.140637 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:14.140600 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:15.086265 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:15.086231 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:15.186132 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:15.186097 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:15.849236 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:15.849182 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:15.949553 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:15.949512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:16.796747 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:16.796709 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:16.906580 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:16.906543 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:17.579554 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:17.579516 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:17.680393 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:17.680358 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:18.499425 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:18.499382 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:18.600145 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:18.600108 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:19.323358 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:19.323320 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:19.421859 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:19.421822 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:20.223533 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:20.223494 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:20.323870 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:20.323831 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:21.397164 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:21.397128 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:21.464297 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:21.464259 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:21.974748 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:21.974711 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:22.075198 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:22.075158 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:22.765072 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:22.765038 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:22.865903 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:22.865860 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:23.834703 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:23.834663 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:24.245928 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:24.245887 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:24.629789 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:24.629685 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:24.729764 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:24.729721 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:25.925021 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:25.924983 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:26.025156 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:26.025120 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:26.419409 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:26.419374 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:26.520352 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:26.520307 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:26.986710 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:26.986671 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:47:27.512433 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:27.512392 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:27.615528 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:27.615483 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:28.455129 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:28.455089 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:28.850873 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:28.850771 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:29.531081 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:29.531035 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:29.633347 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:29.633308 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:30.452899 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:30.452852 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:30.547132 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:30.547094 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:31.449121 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:31.449084 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:31.551360 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:31.551318 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:32.212255 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:32.212196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:32.310737 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:32.310699 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:33.158694 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:33.158655 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:33.257471 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:33.257432 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:33.998832 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:33.998786 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:34.148192 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:34.148156 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:34.797816 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:34.797776 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:34.895926 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:34.895886 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:35.781672 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:35.781626 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:35.881118 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:35.881071 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:36.603090 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:36.603043 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:36.704615 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:36.704576 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:37.432653 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:37.432611 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:37.531156 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:37.531116 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:38.495119 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:38.495077 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:38.596510 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:38.596471 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:39.346298 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:39.346243 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:39.447469 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:39.447429 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:40.472735 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:40.472695 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:40.572788 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:40.572748 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:41.030000 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:41.029955 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:41.130306 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:41.130265 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:42.124725 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:42.124684 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:43.462636 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:43.462598 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:43.832678 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:43.832574 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:43.929612 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:43.929577 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:44.574637 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:44.574599 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:44.674938 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:44.674904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:45.491737 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:45.491686 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:45.559068 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:45.559028 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:47:45.561457 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:45.561421 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:46.369917 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:46.369870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:46.472539 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:46.472500 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:47.387531 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:47.387489 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:47.487260 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:47.487194 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:48.186689 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:48.186647 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:48.287175 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:48.287134 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:49.122899 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:49.122857 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:49.223456 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:49.223420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:50.194291 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:50.194250 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd"] Jul 23 18:47:50.233770 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:50.233726 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:50.393854 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:50.393819 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:50.534776 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:50.534729 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:50.974959 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:50.974901 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8"] Jul 23 18:47:50.997432 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:50.997389 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:51.346134 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:51.346026 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:51.445864 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:51.445832 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:52.155815 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:52.155775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:52.259034 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:52.258995 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:53.014484 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:53.014444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:53.121367 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:53.121327 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:53.813251 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:53.813193 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:53.915313 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:53.915278 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:54.753873 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:54.753831 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:54.854913 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:54.854867 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:55.649442 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:55.649406 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:55.749342 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:55.749292 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:56.425472 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:56.425433 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:56.526990 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:56.526954 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:57.422124 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:57.422086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:57.522920 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:57.522882 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:58.207561 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:58.207523 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:58.308292 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:58.308252 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:59.224383 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:59.224341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:59.326510 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:47:59.326475 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:00.180140 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:00.180101 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:00.280841 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:00.280804 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:01.214662 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:01.214622 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:01.305108 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:01.305071 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:02.023709 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:02.023675 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:02.124251 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:02.124199 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:02.453608 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:02.453573 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:48:03.008194 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:03.008160 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:03.116810 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:03.116768 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:03.872240 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:03.872178 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:03.972476 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:03.972439 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:04.879502 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:04.879468 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:04.981942 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:04.981897 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:05.215102 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.215064 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" podUID="d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" containerName="main" containerID="cri-o://9487d1bcf03854204e3ae88fadbedd66ae8a9310dd9ba4733cee47e4a3362009" gracePeriod=45 Jul 23 18:48:05.375275 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.375242 2562 generic.go:356] "Generic (PLEG): container finished" podID="d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" containerID="9487d1bcf03854204e3ae88fadbedd66ae8a9310dd9ba4733cee47e4a3362009" exitCode=0 Jul 23 18:48:05.375455 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.375294 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" event={"ID":"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9","Type":"ContainerDied","Data":"9487d1bcf03854204e3ae88fadbedd66ae8a9310dd9ba4733cee47e4a3362009"} Jul 23 18:48:05.473102 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.473030 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:48:05.573801 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.573763 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-665rd\" (UniqueName: \"kubernetes.io/projected/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-kube-api-access-665rd\") pod \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " Jul 23 18:48:05.574006 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.573816 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-home\") pod \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " Jul 23 18:48:05.574006 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.573843 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tmp-dir\") pod \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " Jul 23 18:48:05.574006 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.573921 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-dshm\") pod \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " Jul 23 18:48:05.574006 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.573973 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tls-certs\") pod \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " Jul 23 18:48:05.574006 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.573991 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-model-cache\") pod \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\" (UID: \"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9\") " Jul 23 18:48:05.574338 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.574109 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-home" (OuterVolumeSpecName: "home") pod "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" (UID: "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:48:05.574338 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.574156 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" (UID: "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:48:05.574438 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.574351 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-home\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:05.574438 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.574348 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-model-cache" (OuterVolumeSpecName: "model-cache") pod "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" (UID: "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:48:05.574438 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.574371 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tmp-dir\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:05.576268 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.576234 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" (UID: "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:48:05.576268 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.576244 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-kube-api-access-665rd" (OuterVolumeSpecName: "kube-api-access-665rd") pod "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" (UID: "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9"). InnerVolumeSpecName "kube-api-access-665rd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:48:05.576429 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.576290 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-dshm" (OuterVolumeSpecName: "dshm") pod "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" (UID: "d9ac055c-fd6d-4a70-97e4-8f260cfbafc9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:48:05.675489 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.675455 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-tls-certs\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:05.675489 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.675485 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-model-cache\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:05.675489 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.675495 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-665rd\" (UniqueName: \"kubernetes.io/projected/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-kube-api-access-665rd\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:05.675754 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.675504 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9-dshm\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:05.760734 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.760638 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:05.860598 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.860559 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:05.998003 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:05.997960 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" podUID="a8c078ec-1347-42b5-8e1a-c3196806051f" containerName="main" containerID="cri-o://5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498" gracePeriod=45 Jul 23 18:48:06.252882 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.252856 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:48:06.282036 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.282000 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-model-cache\") pod \"a8c078ec-1347-42b5-8e1a-c3196806051f\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " Jul 23 18:48:06.282230 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.282094 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-tmp-dir\") pod \"a8c078ec-1347-42b5-8e1a-c3196806051f\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " Jul 23 18:48:06.282230 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.282133 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49nwk\" (UniqueName: \"kubernetes.io/projected/a8c078ec-1347-42b5-8e1a-c3196806051f-kube-api-access-49nwk\") pod \"a8c078ec-1347-42b5-8e1a-c3196806051f\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " Jul 23 18:48:06.282230 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.282191 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-dshm\") pod \"a8c078ec-1347-42b5-8e1a-c3196806051f\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " Jul 23 18:48:06.282384 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.282249 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c078ec-1347-42b5-8e1a-c3196806051f-tls-certs\") pod \"a8c078ec-1347-42b5-8e1a-c3196806051f\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " Jul 23 18:48:06.282384 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.282301 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-home\") pod \"a8c078ec-1347-42b5-8e1a-c3196806051f\" (UID: \"a8c078ec-1347-42b5-8e1a-c3196806051f\") " Jul 23 18:48:06.282384 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.282302 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-model-cache" (OuterVolumeSpecName: "model-cache") pod "a8c078ec-1347-42b5-8e1a-c3196806051f" (UID: "a8c078ec-1347-42b5-8e1a-c3196806051f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:48:06.282547 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.282393 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a8c078ec-1347-42b5-8e1a-c3196806051f" (UID: "a8c078ec-1347-42b5-8e1a-c3196806051f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:48:06.282624 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.282599 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-model-cache\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:06.282624 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.282611 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-home" (OuterVolumeSpecName: "home") pod "a8c078ec-1347-42b5-8e1a-c3196806051f" (UID: "a8c078ec-1347-42b5-8e1a-c3196806051f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:48:06.282624 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.282623 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-tmp-dir\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:06.284525 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.284494 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-dshm" (OuterVolumeSpecName: "dshm") pod "a8c078ec-1347-42b5-8e1a-c3196806051f" (UID: "a8c078ec-1347-42b5-8e1a-c3196806051f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:48:06.284525 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.284495 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8c078ec-1347-42b5-8e1a-c3196806051f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a8c078ec-1347-42b5-8e1a-c3196806051f" (UID: "a8c078ec-1347-42b5-8e1a-c3196806051f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:48:06.284692 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.284588 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c078ec-1347-42b5-8e1a-c3196806051f-kube-api-access-49nwk" (OuterVolumeSpecName: "kube-api-access-49nwk") pod "a8c078ec-1347-42b5-8e1a-c3196806051f" (UID: "a8c078ec-1347-42b5-8e1a-c3196806051f"). InnerVolumeSpecName "kube-api-access-49nwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:48:06.382093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.381995 2562 generic.go:356] "Generic (PLEG): container finished" podID="a8c078ec-1347-42b5-8e1a-c3196806051f" containerID="5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498" exitCode=0 Jul 23 18:48:06.382093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.382039 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" event={"ID":"a8c078ec-1347-42b5-8e1a-c3196806051f","Type":"ContainerDied","Data":"5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498"} Jul 23 18:48:06.382093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.382048 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" Jul 23 18:48:06.382441 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.382097 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8" event={"ID":"a8c078ec-1347-42b5-8e1a-c3196806051f","Type":"ContainerDied","Data":"2132bdfc15c8d71d75161c61faa3a908ad01b16d63c331f4f53e91bd73bbf861"} Jul 23 18:48:06.382441 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.382121 2562 scope.go:117] "RemoveContainer" containerID="5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498" Jul 23 18:48:06.383148 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.383113 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-home\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:06.383321 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.383150 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49nwk\" (UniqueName: \"kubernetes.io/projected/a8c078ec-1347-42b5-8e1a-c3196806051f-kube-api-access-49nwk\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:06.383321 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.383165 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c078ec-1347-42b5-8e1a-c3196806051f-dshm\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:06.383321 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.383179 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c078ec-1347-42b5-8e1a-c3196806051f-tls-certs\") on node \"ip-10-0-129-187.ec2.internal\" DevicePath \"\"" Jul 23 18:48:06.383728 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.383707 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" Jul 23 18:48:06.383866 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.383732 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd" event={"ID":"d9ac055c-fd6d-4a70-97e4-8f260cfbafc9","Type":"ContainerDied","Data":"53a204d12061ac229ba410e72ef39e7963dcdba7ebe718e83fd550b3e492854f"} Jul 23 18:48:06.393157 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.393138 2562 scope.go:117] "RemoveContainer" containerID="5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498" Jul 23 18:48:06.393484 ip-10-0-129-187 kubenswrapper[2562]: E0723 18:48:06.393467 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498\": container with ID starting with 5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498 not found: ID does not exist" containerID="5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498" Jul 23 18:48:06.393542 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.393501 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498"} err="failed to get container status \"5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498\": rpc error: code = NotFound desc = could not find container \"5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498\": container with ID starting with 5bb8edb20c3336db69b6cb3d5cf4d2d17b0a104819b2474d8dab92db5892f498 not found: ID does not exist" Jul 23 18:48:06.393542 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.393518 2562 scope.go:117] "RemoveContainer" containerID="9487d1bcf03854204e3ae88fadbedd66ae8a9310dd9ba4733cee47e4a3362009" Jul 23 18:48:06.409614 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.409570 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8"] Jul 23 18:48:06.417819 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.417777 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8ee985/rate-test-model-8ee985-kserve-bb646cdc5-hntg8"] Jul 23 18:48:06.429724 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.429694 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd"] Jul 23 18:48:06.434885 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.434857 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5f06b1/rate-test-model-5f06b1-kserve-5987fc464c-thczd"] Jul 23 18:48:06.704763 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.704722 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:06.804887 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:06.804852 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:07.130832 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:07.130732 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8c078ec-1347-42b5-8e1a-c3196806051f" path="/var/lib/kubelet/pods/a8c078ec-1347-42b5-8e1a-c3196806051f/volumes" Jul 23 18:48:07.131197 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:07.131138 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" path="/var/lib/kubelet/pods/d9ac055c-fd6d-4a70-97e4-8f260cfbafc9/volumes" Jul 23 18:48:07.617922 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:07.617881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:07.719987 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:07.719948 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:08.705092 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:08.705007 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:08.808701 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:08.808667 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:09.428194 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:09.428158 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:09.527228 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:09.527188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:10.313851 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:10.313812 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:10.413375 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:10.413341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:11.118060 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:11.118027 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:11.219174 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:11.219138 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:12.078277 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:12.078241 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:12.152915 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:12.152883 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:12.796138 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:12.796107 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:12.896168 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:12.896136 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:13.696283 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:13.696182 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:13.795774 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:13.795739 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:14.465826 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:14.465785 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:14.566605 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:14.566568 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:15.267535 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:15.267499 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:15.368278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:15.368240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:16.072370 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:16.072327 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:16.171467 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:16.171431 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:16.990581 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:16.990548 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:17.091675 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:17.091625 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:17.779965 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:17.779923 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:17.881208 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:17.881171 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:18.841642 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:18.841615 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:18.942750 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:18.942706 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:19.603648 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:19.603612 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:19.709188 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:19.709155 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:20.473261 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:20.473201 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:20.573993 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:20.573956 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:20.783460 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:20.783358 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:21.548890 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:21.548854 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:21.654338 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:21.654298 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:22.622271 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:22.622231 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:22.721700 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:22.721666 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:23.420339 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:23.420295 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:23.521809 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:23.521768 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:23.960326 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:23.960286 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:48:24.453477 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:24.453438 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:24.561746 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:24.561710 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:25.382003 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:25.381965 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:25.483396 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:25.483360 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:25.631418 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:25.631380 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:26.075468 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:26.075428 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:26.431260 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:26.431143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:26.533989 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:26.533948 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:27.217836 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:27.217800 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:27.318754 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:27.318708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:28.080766 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:28.080731 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:28.181312 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:28.181276 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:28.911741 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:28.911703 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:29.013582 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:29.013537 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:29.890221 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:29.890186 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:29.990002 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:29.989960 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:30.845432 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:30.845395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:30.945620 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:30.945574 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:31.808515 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:31.808466 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:31.910866 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:31.910826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:32.488273 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:32.488207 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:32.591162 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:32.591123 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:32.712754 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:32.712717 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:32.815107 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:32.815006 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:32.984586 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:32.984546 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:33.083286 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:33.083169 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:33.974352 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:33.974320 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:34.072559 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:34.072524 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:34.864487 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:34.864455 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:34.964884 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:34.964850 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:35.740268 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:35.740228 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:35.839168 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:35.839130 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:36.553281 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:36.553245 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:36.657919 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:36.657886 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:37.400128 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:37.400096 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:37.502518 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:37.502472 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:38.242121 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:38.242077 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:38.346178 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:38.346138 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:39.240636 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:39.240599 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:39.342572 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:39.342538 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:40.066036 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:40.065998 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:40.166487 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:40.166449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:40.964440 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:40.964404 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:41.066938 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:41.066901 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:41.841916 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:41.841881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:41.942302 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:41.942265 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:42.716229 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:42.716173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:42.816404 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:42.816363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:43.745133 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:43.745090 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:43.867222 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:43.867185 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:44.729917 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:44.729856 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:44.830311 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:44.830277 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:45.002494 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:45.002399 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:47.963967 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:47.963930 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-q58gk"] Jul 23 18:48:48.349377 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:48.349289 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:48.748983 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:48.748950 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:55.386867 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:55.386831 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:55.488180 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:55.488143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:55.584614 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:55.584575 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:55.687621 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:55.687532 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:55.930689 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:55.930657 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:48:56.413040 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:48:56.413003 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6ddffd9688-gm2xx_5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6/authorino/0.log" Jul 23 18:49:03.079420 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:03.079332 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657dbb4b4c-rkhxt_d2e60fe4-ac96-4576-8cba-da52a5e531eb/copy-manifests/0.log" Jul 23 18:49:03.098223 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:03.098173 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657dbb4b4c-rkhxt_d2e60fe4-ac96-4576-8cba-da52a5e531eb/manager/0.log" Jul 23 18:49:05.523393 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:05.523355 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6ddffd9688-gm2xx_5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6/authorino/0.log" Jul 23 18:49:06.290278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:06.290237 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-q58gk_e16edf74-ea69-4860-9043-b94ee9b4da16/limitador/0.log" Jul 23 18:49:07.066529 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:07.066497 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d9cbf8b4-w6kwv_3598fbbd-f46f-4e8e-9782-b49dd9ec1c37/kube-auth-proxy/0.log" Jul 23 18:49:08.552100 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:08.552058 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-s2x6g_22778797-4630-465c-896d-5b7e42b44bef/main/0.log" Jul 23 18:49:08.674757 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:08.674721 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-gdwxj_be16716d-7764-43ba-aed9-3c795eff521c/main/0.log" Jul 23 18:49:14.277042 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:14.277004 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:49:14.381039 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:14.380992 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" limit=1024 Jul 23 18:49:17.779979 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:17.779943 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h62kj_822c461b-5e7b-4abb-a090-edacc6b4392c/global-pull-secret-syncer/0.log" Jul 23 18:49:17.887206 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:17.887170 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-45szx_14934d93-f8ca-478d-ba2d-24c70d9ab9ec/konnectivity-agent/0.log" Jul 23 18:49:17.993288 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:17.993250 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-187.ec2.internal_12a6046fa193489a163cd70c7bb12503/haproxy/0.log" Jul 23 18:49:22.936206 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:22.936166 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6ddffd9688-gm2xx_5f8a3f95-8ba6-4c78-aad5-3b3bbaaf24c6/authorino/0.log" Jul 23 18:49:23.375168 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:23.375130 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-q58gk_e16edf74-ea69-4860-9043-b94ee9b4da16/limitador/0.log" Jul 23 18:49:25.590197 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:25.590160 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-ck2nn_821d3383-f3b4-4921-a42c-90d1ec4a0993/monitoring-plugin/0.log" Jul 23 18:49:25.813730 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:25.813688 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ppjwf_584ac4dd-5636-4cf6-9f5c-d461f9a3f53a/node-exporter/0.log" Jul 23 18:49:25.833617 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:25.833581 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ppjwf_584ac4dd-5636-4cf6-9f5c-d461f9a3f53a/kube-rbac-proxy/0.log" Jul 23 18:49:25.854876 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:25.854795 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ppjwf_584ac4dd-5636-4cf6-9f5c-d461f9a3f53a/init-textfile/0.log" Jul 23 18:49:25.883965 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:25.883934 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-ngtqp_34cb4db7-dec6-4353-b555-b56201b8d2e0/kube-rbac-proxy-main/0.log" Jul 23 18:49:25.907727 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:25.907695 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-ngtqp_34cb4db7-dec6-4353-b555-b56201b8d2e0/kube-rbac-proxy-self/0.log" Jul 23 18:49:25.933885 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:25.933845 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-ngtqp_34cb4db7-dec6-4353-b555-b56201b8d2e0/openshift-state-metrics/0.log" Jul 23 18:49:26.169637 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:26.169537 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-7pzkp_52885627-f5ff-42a6-b040-6e1e0c89934a/prometheus-operator/0.log" Jul 23 18:49:26.186377 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:26.186346 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-7pzkp_52885627-f5ff-42a6-b040-6e1e0c89934a/kube-rbac-proxy/0.log" Jul 23 18:49:26.260371 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:26.260341 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6659667f4d-2gnhv_1dff4e18-e2f0-4157-b487-6dcb65870f0b/telemeter-client/0.log" Jul 23 18:49:26.285995 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:26.285948 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6659667f4d-2gnhv_1dff4e18-e2f0-4157-b487-6dcb65870f0b/reload/0.log" Jul 23 18:49:26.321276 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:26.321245 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6659667f4d-2gnhv_1dff4e18-e2f0-4157-b487-6dcb65870f0b/kube-rbac-proxy/0.log" Jul 23 18:49:26.366384 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:26.366351 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b678b85c5-5ksjg_390867ea-0d3a-48fc-b960-d961f286c476/thanos-query/0.log" Jul 23 18:49:26.389077 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:26.389049 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b678b85c5-5ksjg_390867ea-0d3a-48fc-b960-d961f286c476/kube-rbac-proxy-web/0.log" Jul 23 18:49:26.411873 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:26.411836 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b678b85c5-5ksjg_390867ea-0d3a-48fc-b960-d961f286c476/kube-rbac-proxy/0.log" Jul 23 18:49:26.434419 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:26.434316 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b678b85c5-5ksjg_390867ea-0d3a-48fc-b960-d961f286c476/prom-label-proxy/0.log" Jul 23 18:49:26.470124 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:26.470096 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b678b85c5-5ksjg_390867ea-0d3a-48fc-b960-d961f286c476/kube-rbac-proxy-rules/0.log" Jul 23 18:49:26.492464 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:26.492431 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b678b85c5-5ksjg_390867ea-0d3a-48fc-b960-d961f286c476/kube-rbac-proxy-metrics/0.log" Jul 23 18:49:27.458093 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.458049 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh"] Jul 23 18:49:27.458527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.458468 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8c078ec-1347-42b5-8e1a-c3196806051f" containerName="main" Jul 23 18:49:27.458527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.458481 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8c078ec-1347-42b5-8e1a-c3196806051f" containerName="main" Jul 23 18:49:27.458527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.458493 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" containerName="main" Jul 23 18:49:27.458527 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.458499 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" containerName="main" Jul 23 18:49:27.458665 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.458563 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8c078ec-1347-42b5-8e1a-c3196806051f" containerName="main" Jul 23 18:49:27.458665 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.458573 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9ac055c-fd6d-4a70-97e4-8f260cfbafc9" containerName="main" Jul 23 18:49:27.461710 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.461692 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.464373 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.464349 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cpdkp\"/\"default-dockercfg-m9lj7\"" Jul 23 18:49:27.465229 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.465181 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cpdkp\"/\"openshift-service-ca.crt\"" Jul 23 18:49:27.465358 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.465248 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cpdkp\"/\"kube-root-ca.crt\"" Jul 23 18:49:27.473362 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.473335 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh"] Jul 23 18:49:27.606472 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.606431 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-lib-modules\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.606677 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.606515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-podres\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.606677 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.606603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-proc\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.606677 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.606625 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-sys\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.606677 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.606645 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj9v6\" (UniqueName: \"kubernetes.io/projected/348462f0-dd8b-4712-b405-cc66f5e9757d-kube-api-access-zj9v6\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.707872 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.707831 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-podres\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.708091 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.707926 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-proc\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.708091 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.707945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-sys\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.708091 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.707967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj9v6\" (UniqueName: \"kubernetes.io/projected/348462f0-dd8b-4712-b405-cc66f5e9757d-kube-api-access-zj9v6\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.708091 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.707992 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-lib-modules\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.708091 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.708029 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-podres\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.708091 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.708053 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-sys\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.708091 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.708074 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-proc\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.708413 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.708161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/348462f0-dd8b-4712-b405-cc66f5e9757d-lib-modules\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.717238 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.717185 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj9v6\" (UniqueName: \"kubernetes.io/projected/348462f0-dd8b-4712-b405-cc66f5e9757d-kube-api-access-zj9v6\") pod \"perf-node-gather-daemonset-lzsdh\" (UID: \"348462f0-dd8b-4712-b405-cc66f5e9757d\") " pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.739960 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.739931 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-jhzbd_4efa0791-cff4-4eef-9f3a-2a584c874ba5/networking-console-plugin/0.log" Jul 23 18:49:27.772282 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.772249 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:27.909594 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:27.909536 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh"] Jul 23 18:49:27.911975 ip-10-0-129-187 kubenswrapper[2562]: W0723 18:49:27.911936 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod348462f0_dd8b_4712_b405_cc66f5e9757d.slice/crio-5ef8f93fe3ae26c6d90575e691f563190171344e430e13bcb329b428711486ea WatchSource:0}: Error finding container 5ef8f93fe3ae26c6d90575e691f563190171344e430e13bcb329b428711486ea: Status 404 returned error can't find the container with id 5ef8f93fe3ae26c6d90575e691f563190171344e430e13bcb329b428711486ea Jul 23 18:49:28.372513 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:28.372412 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:49:28.383951 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:28.383900 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/2.log" Jul 23 18:49:28.798578 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:28.798536 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" event={"ID":"348462f0-dd8b-4712-b405-cc66f5e9757d","Type":"ContainerStarted","Data":"84dad459829e3d78ea04139a20937d414c1a18d7736a7e193b0f4ccadd9b1f4d"} Jul 23 18:49:28.799060 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:28.798594 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" event={"ID":"348462f0-dd8b-4712-b405-cc66f5e9757d","Type":"ContainerStarted","Data":"5ef8f93fe3ae26c6d90575e691f563190171344e430e13bcb329b428711486ea"} Jul 23 18:49:28.799060 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:28.798642 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:28.816586 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:28.816524 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" podStartSLOduration=1.816507069 podStartE2EDuration="1.816507069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:49:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:49:28.815268141 +0000 UTC m=+2990.302760130" watchObservedRunningTime="2026-07-23 18:49:28.816507069 +0000 UTC m=+2990.303999039" Jul 23 18:49:30.768739 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:30.768703 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hzc9d_84ca6f4e-056f-4d1f-93a4-2e910ef06f08/dns/0.log" Jul 23 18:49:30.791808 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:30.791782 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hzc9d_84ca6f4e-056f-4d1f-93a4-2e910ef06f08/kube-rbac-proxy/0.log" Jul 23 18:49:30.933925 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:30.933897 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngf5j_b673b420-6fcb-4021-8ea7-1dce6101f1ad/dns-node-resolver/0.log" Jul 23 18:49:31.508873 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:31.508838 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-65749cb4f-4xvjg_02000744-cf65-4f6e-8f67-40b30f0d9cc3/registry/0.log" Jul 23 18:49:31.566391 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:31.566360 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ljb4k_b939e580-7ce8-4e0b-a2c6-6e55f65e25d8/node-ca/0.log" Jul 23 18:49:32.824087 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:32.824055 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d9cbf8b4-w6kwv_3598fbbd-f46f-4e8e-9782-b49dd9ec1c37/kube-auth-proxy/0.log" Jul 23 18:49:33.716983 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:33.716954 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fsxnc_0833f914-e35c-4a6d-8a60-0e99269bee04/serve-healthcheck-canary/0.log" Jul 23 18:49:34.289269 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:34.289241 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2snwn_ccdc7fc1-9212-4c7f-ad21-3916e5496d12/kube-rbac-proxy/0.log" Jul 23 18:49:34.311307 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:34.311276 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2snwn_ccdc7fc1-9212-4c7f-ad21-3916e5496d12/exporter/0.log" Jul 23 18:49:34.337478 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:34.337445 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2snwn_ccdc7fc1-9212-4c7f-ad21-3916e5496d12/extractor/0.log" Jul 23 18:49:34.813921 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:34.813891 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cpdkp/perf-node-gather-daemonset-lzsdh" Jul 23 18:49:37.527830 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:37.527789 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657dbb4b4c-rkhxt_d2e60fe4-ac96-4576-8cba-da52a5e531eb/manager/0.log" Jul 23 18:49:37.567292 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:37.567202 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657dbb4b4c-rkhxt_d2e60fe4-ac96-4576-8cba-da52a5e531eb/copy-manifests/0.log" Jul 23 18:49:38.528946 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:38.528914 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d9cbf8b4-w6kwv_3598fbbd-f46f-4e8e-9782-b49dd9ec1c37/kube-auth-proxy/0.log" Jul 23 18:49:39.058444 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:39.058355 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7ff5b44f4d-nrgnq_63eb0400-1403-45be-8d95-ea1aee7aa740/manager/0.log" Jul 23 18:49:39.084881 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:39.084846 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-7tt9x_d2de6c0f-4a95-4100-a495-47e47c7d1575/openshift-lws-operator/0.log" Jul 23 18:49:39.220607 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:39.220580 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:49:39.227547 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:39.227525 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qgsr_ff788e6b-0522-4742-8f8f-d100924612a0/console-operator/1.log" Jul 23 18:49:40.091278 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:40.091246 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d9cbf8b4-w6kwv_3598fbbd-f46f-4e8e-9782-b49dd9ec1c37/kube-auth-proxy/0.log" Jul 23 18:49:44.584072 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:44.584029 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-wgqrh_8152bbb3-da20-460a-b6e7-2a5aa61bee78/kube-storage-version-migrator-operator/1.log" Jul 23 18:49:44.586085 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:44.586057 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-wgqrh_8152bbb3-da20-460a-b6e7-2a5aa61bee78/kube-storage-version-migrator-operator/0.log" Jul 23 18:49:45.616181 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:45.616146 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5m2x2_1b816e2d-d250-4c9f-abb2-ff37218ff860/kube-multus-additional-cni-plugins/0.log" Jul 23 18:49:45.637850 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:45.637816 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5m2x2_1b816e2d-d250-4c9f-abb2-ff37218ff860/egress-router-binary-copy/0.log" Jul 23 18:49:45.659089 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:45.659069 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5m2x2_1b816e2d-d250-4c9f-abb2-ff37218ff860/cni-plugins/0.log" Jul 23 18:49:45.679999 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:45.679973 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5m2x2_1b816e2d-d250-4c9f-abb2-ff37218ff860/bond-cni-plugin/0.log" Jul 23 18:49:45.702677 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:45.702645 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5m2x2_1b816e2d-d250-4c9f-abb2-ff37218ff860/routeoverride-cni/0.log" Jul 23 18:49:45.729724 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:45.729701 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5m2x2_1b816e2d-d250-4c9f-abb2-ff37218ff860/whereabouts-cni-bincopy/0.log" Jul 23 18:49:45.751878 ip-10-0-129-187 kubenswrapper[2562]: I0723 18:49:45.751851 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5m2x2_1b816e2d-d250-4c9f-abb2-ff37218ff860/whereabouts-cni/0.log"