Jul 05 11:51:34.934822 ip-10-0-142-207 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 05 11:51:34.934831 ip-10-0-142-207 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 05 11:51:34.934838 ip-10-0-142-207 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 05 11:51:34.935074 ip-10-0-142-207 systemd[1]: Failed to start Kubernetes Kubelet. Jul 05 11:51:45.181614 ip-10-0-142-207 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 05 11:51:45.181633 ip-10-0-142-207 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 6e06cb45c5f744c89cbd171c59f267f2 -- Jul 05 11:53:19.495235 ip-10-0-142-207 systemd[1]: Starting Kubernetes Kubelet... Jul 05 11:53:19.926969 ip-10-0-142-207 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 11:53:19.926969 ip-10-0-142-207 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 05 11:53:19.926969 ip-10-0-142-207 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 11:53:19.926969 ip-10-0-142-207 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 05 11:53:19.926969 ip-10-0-142-207 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 11:53:19.928053 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.927964 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 05 11:53:19.932897 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932863 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:19.932897 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932894 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:19.932897 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932899 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:19.932897 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932903 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932907 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932912 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932915 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932922 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932926 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932930 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932938 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932943 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932947 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932951 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932955 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932959 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932965 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932971 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932975 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932979 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932984 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932988 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:19.933126 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932992 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.932996 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933001 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933004 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933009 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933013 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933018 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933022 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933028 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933032 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933036 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933040 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933044 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933048 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933052 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933060 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933068 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933072 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933078 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933082 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:19.933722 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933086 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933090 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933095 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933099 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933104 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933108 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933113 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933117 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933121 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933125 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933130 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933134 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933139 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933143 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933148 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933152 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933156 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933160 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933164 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933168 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:19.934254 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933175 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933179 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933184 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933188 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933192 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933197 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933201 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933206 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933211 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933217 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933221 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933226 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933230 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933234 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933239 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933243 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933247 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933251 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933255 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933260 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:19.935093 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933264 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933268 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933272 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933276 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933280 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933284 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933288 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933292 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933297 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933301 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933305 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933309 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933313 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933320 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933325 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933330 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.933334 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935384 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935395 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:19.935953 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935402 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935407 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935412 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935418 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935422 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935427 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935431 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935435 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935439 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935443 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935447 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935452 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935458 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935464 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935469 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935476 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935482 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935489 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935494 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:19.936728 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935499 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935504 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935509 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935513 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935517 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935523 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935528 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935532 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935536 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935541 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935545 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935549 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935553 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935557 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935561 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935566 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935570 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935575 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935579 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:19.937602 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935583 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935588 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935592 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935596 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935600 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935604 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935608 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935613 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935617 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935622 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935626 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935630 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935634 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935638 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935643 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935647 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935651 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935656 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935660 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935664 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:19.938336 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935668 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935672 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935676 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935680 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935684 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935688 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935692 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935696 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935701 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935706 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935713 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935719 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935723 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935728 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935732 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935736 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935741 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935745 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935749 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935753 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:19.938819 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935757 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935763 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935767 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935772 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935776 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935781 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935785 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935790 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935794 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935798 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935802 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935806 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935810 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935815 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935819 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935823 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935827 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935832 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.935836 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936467 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:19.939339 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936476 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936481 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936485 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936491 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936496 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936500 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936508 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936514 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936519 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936524 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936528 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936533 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936539 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936544 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936548 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936553 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936557 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936561 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936566 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:19.939817 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936570 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936575 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936579 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936583 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936588 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936592 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936596 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936600 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936604 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936608 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936613 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936617 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936621 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936625 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936630 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936634 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936638 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936648 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936652 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936658 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:19.940286 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936662 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936667 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936671 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936675 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936680 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936684 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936689 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936693 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936698 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936702 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936706 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936711 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936714 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936719 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936723 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936727 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936731 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936735 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936739 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936743 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:19.940773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936746 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936750 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936754 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936759 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936763 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936768 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936772 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936776 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936780 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936784 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936789 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936793 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936798 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936802 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936806 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936810 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936815 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936821 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936825 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:19.941268 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936831 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936837 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936843 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936847 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936851 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936855 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936861 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936866 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936890 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936897 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936902 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936906 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936911 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936915 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936920 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936924 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936928 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936932 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936937 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:19.941712 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.936941 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937045 2575 flags.go:64] FLAG: --address="0.0.0.0" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937067 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937077 2575 flags.go:64] FLAG: --anonymous-auth="true" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937084 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937092 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937097 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937104 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937111 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937116 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937121 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937128 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937135 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937141 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937146 2575 flags.go:64] FLAG: --cgroup-root="" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937150 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937155 2575 flags.go:64] FLAG: --client-ca-file="" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937160 2575 flags.go:64] FLAG: --cloud-provider="external" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937165 2575 flags.go:64] FLAG: --cluster-dns="[]" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937171 2575 flags.go:64] FLAG: --cluster-domain="" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937176 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937180 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937185 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937191 2575 flags.go:64] FLAG: --container-log-max-files="5" Jul 05 11:53:19.942193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937199 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937205 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937210 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937216 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937220 2575 flags.go:64] FLAG: --contention-profiling="false" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937225 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937230 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937235 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937240 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937246 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937251 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937257 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937262 2575 flags.go:64] FLAG: --enable-load-reader="false" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937266 2575 flags.go:64] FLAG: --enable-server="true" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937271 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937279 2575 flags.go:64] FLAG: --event-burst="100" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937284 2575 flags.go:64] FLAG: --event-qps="50" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937289 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937294 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937301 2575 flags.go:64] FLAG: --eviction-hard="" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937307 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937312 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937316 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937321 2575 flags.go:64] FLAG: --eviction-soft="" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937326 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 05 11:53:19.942781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937331 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937336 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937340 2575 flags.go:64] FLAG: --experimental-mounter-path="" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937345 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937350 2575 flags.go:64] FLAG: --fail-swap-on="true" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937354 2575 flags.go:64] FLAG: --feature-gates="" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937361 2575 flags.go:64] FLAG: --file-check-frequency="20s" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937366 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937371 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937376 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937380 2575 flags.go:64] FLAG: --healthz-port="10248" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937385 2575 flags.go:64] FLAG: --help="false" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937390 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-142-207.ec2.internal" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937395 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937400 2575 flags.go:64] FLAG: --http-check-frequency="20s" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937404 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937410 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937416 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937421 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937425 2575 flags.go:64] FLAG: --image-service-endpoint="" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937430 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937435 2575 flags.go:64] FLAG: --kube-api-burst="100" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937440 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937445 2575 flags.go:64] FLAG: --kube-api-qps="50" Jul 05 11:53:19.943453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937450 2575 flags.go:64] FLAG: --kube-reserved="" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937455 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937461 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937466 2575 flags.go:64] FLAG: --kubelet-cgroups="" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937471 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937475 2575 flags.go:64] FLAG: --lock-file="" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937480 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937485 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937489 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937499 2575 flags.go:64] FLAG: --log-json-split-stream="false" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937504 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937508 2575 flags.go:64] FLAG: --log-text-split-stream="false" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937513 2575 flags.go:64] FLAG: --logging-format="text" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937517 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937523 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937528 2575 flags.go:64] FLAG: --manifest-url="" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937533 2575 flags.go:64] FLAG: --manifest-url-header="" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937539 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937544 2575 flags.go:64] FLAG: --max-open-files="1000000" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937551 2575 flags.go:64] FLAG: --max-pods="110" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937556 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937560 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937565 2575 flags.go:64] FLAG: --memory-manager-policy="None" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937570 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937575 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 05 11:53:19.944064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937580 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937584 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937595 2575 flags.go:64] FLAG: --node-status-max-images="50" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937600 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937604 2575 flags.go:64] FLAG: --oom-score-adj="-999" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937609 2575 flags.go:64] FLAG: --pod-cidr="" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937614 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937622 2575 flags.go:64] FLAG: --pod-manifest-path="" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937627 2575 flags.go:64] FLAG: --pod-max-pids="-1" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937632 2575 flags.go:64] FLAG: --pods-per-core="0" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937639 2575 flags.go:64] FLAG: --port="10250" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937644 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937649 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0326806485f43d2b6" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937655 2575 flags.go:64] FLAG: --qos-reserved="" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937659 2575 flags.go:64] FLAG: --read-only-port="10255" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937664 2575 flags.go:64] FLAG: --register-node="true" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937669 2575 flags.go:64] FLAG: --register-with-taints="" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937675 2575 flags.go:64] FLAG: --registry-burst="10" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937679 2575 flags.go:64] FLAG: --registry-qps="5" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937684 2575 flags.go:64] FLAG: --reserved-cpus="" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937689 2575 flags.go:64] FLAG: --reserved-memory="" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937695 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937700 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937704 2575 flags.go:64] FLAG: --rotate-certificates="false" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937709 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 05 11:53:19.944688 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937714 2575 flags.go:64] FLAG: --runonce="false" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937719 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937724 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937729 2575 flags.go:64] FLAG: --seccomp-default="false" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937733 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937738 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937743 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937747 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937752 2575 flags.go:64] FLAG: --storage-driver-password="root" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937757 2575 flags.go:64] FLAG: --storage-driver-secure="false" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937762 2575 flags.go:64] FLAG: --storage-driver-table="stats" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937767 2575 flags.go:64] FLAG: --storage-driver-user="root" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937771 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937778 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937784 2575 flags.go:64] FLAG: --system-cgroups="" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937788 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937798 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937804 2575 flags.go:64] FLAG: --tls-cert-file="" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937809 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937816 2575 flags.go:64] FLAG: --tls-min-version="" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937820 2575 flags.go:64] FLAG: --tls-private-key-file="" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937825 2575 flags.go:64] FLAG: --topology-manager-policy="none" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937831 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937835 2575 flags.go:64] FLAG: --topology-manager-scope="container" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937840 2575 flags.go:64] FLAG: --v="2" Jul 05 11:53:19.945339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937846 2575 flags.go:64] FLAG: --version="false" Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937853 2575 flags.go:64] FLAG: --vmodule="" Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937858 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.937864 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938061 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938068 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938074 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938079 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938084 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938088 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938092 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938096 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938101 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938132 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938146 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938150 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938155 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938159 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938162 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938165 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938168 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:19.946004 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938171 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938174 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938177 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938179 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938182 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938185 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938188 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938190 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938193 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938196 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938199 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938201 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938204 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938207 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938210 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938216 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938219 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938221 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938224 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:19.946531 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938226 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938229 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938232 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938235 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938237 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938240 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938244 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938246 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938249 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938251 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938254 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938256 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938259 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938261 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938264 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938266 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938269 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938271 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938274 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:19.947026 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938277 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938279 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938282 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938284 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938287 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938291 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938297 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938299 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938302 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938307 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938309 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938312 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938315 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938318 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938321 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938324 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938326 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938329 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938331 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938334 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:19.947519 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938338 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938341 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938344 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938347 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938351 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938354 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938357 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938360 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938364 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938366 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938369 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938372 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938375 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938378 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938381 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938383 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938386 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938389 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938392 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:19.948017 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938395 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938398 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938402 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938406 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.938410 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.939225 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.946422 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.946441 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.946457 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.946464 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946520 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946525 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946528 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946532 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946535 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:19.948495 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946538 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946541 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946544 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946547 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946550 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946553 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946556 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946558 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946561 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946564 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946566 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946569 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946572 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946574 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946577 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946580 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946583 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946585 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946588 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946591 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:19.949132 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946593 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946596 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946598 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946601 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946605 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946608 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946610 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946613 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946617 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946620 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946623 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946625 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946628 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946630 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946633 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946636 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946638 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946641 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946644 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946646 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:19.949896 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946649 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946652 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946654 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946657 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946660 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946662 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946665 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946667 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946670 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946673 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946675 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946678 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946680 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946683 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946686 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946688 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946691 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946695 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946698 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946700 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:19.950390 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946703 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946706 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946708 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946710 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946715 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946718 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946721 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946723 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946726 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946728 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946731 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946734 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946736 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946739 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946741 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946743 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946746 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946749 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946751 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:19.950858 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946753 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946756 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946760 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946763 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946766 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946769 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946772 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946776 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946778 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946782 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946785 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946788 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946790 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946793 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:19.951387 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946796 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.946801 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946946 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946952 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946955 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946958 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946961 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946963 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946966 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946969 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946971 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946974 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946976 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946979 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946982 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:19.951730 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946985 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946987 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946989 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946992 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946994 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946997 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.946999 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947002 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947004 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947007 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947009 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947012 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947015 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947017 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947020 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947023 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947026 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947028 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947031 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947034 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:19.952111 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947036 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947039 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947042 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947044 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947047 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947049 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947052 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947055 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947060 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947062 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947065 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947068 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947072 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947075 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947077 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947080 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947083 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947085 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947088 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:19.952601 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947090 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947093 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947095 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947098 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947101 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947104 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947107 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947109 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947112 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947115 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947118 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947120 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947123 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947125 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947128 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947130 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947132 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947135 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947137 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:19.953058 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947140 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947142 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947144 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947147 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947149 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947152 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947154 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947157 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947161 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947164 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947167 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947169 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947172 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947174 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947177 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947179 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947182 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947185 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947188 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947190 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:19.953507 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947193 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947195 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947198 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947201 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947203 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947206 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947208 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:19.947211 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.947215 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.947338 2575 server.go:962] "Client rotation is on, will bootstrap in background" Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.950968 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.952756 2575 server.go:1019] "Starting client certificate rotation" Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.952861 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 11:53:19.953989 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.952919 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 11:53:19.955618 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.955604 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 11:53:19.959379 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.959363 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 11:53:19.968943 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.968926 2575 log.go:25] "Validated CRI v1 runtime API" Jul 05 11:53:19.971564 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.971548 2575 log.go:25] "Validated CRI v1 image API" Jul 05 11:53:19.975056 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.975035 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 05 11:53:19.977746 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.977720 2575 fs.go:135] Filesystem UUIDs: map[67b74566-d626-4fdb-9fcb-149f44e3ba79:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 86a67443-53a1-402e-927a-f9a131dd2bd1:/dev/nvme0n1p4] Jul 05 11:53:19.977823 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.977745 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 05 11:53:19.983659 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.983641 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 11:53:19.983816 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.983687 2575 manager.go:216] Machine: {Timestamp:2026-07-05 11:53:19.981326656 +0000 UTC m=+0.372028410 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101127 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25c321043f86513382891abe835460 SystemUUID:ec25c321-043f-8651-3382-891abe835460 BootID:6e06cb45-c5f7-44c8-9cbd-171c59f267f2 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c7:ed:70:96:47 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c7:ed:70:96:47 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:1b:1b:74:bd:b7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 05 11:53:19.983816 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.983813 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 05 11:53:19.983936 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.983914 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 05 11:53:19.987518 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.987482 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 05 11:53:19.987668 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.987521 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-207.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 05 11:53:19.988450 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.988438 2575 topology_manager.go:138] "Creating topology manager with none policy" Jul 05 11:53:19.988486 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.988451 2575 container_manager_linux.go:309] "Creating device plugin manager" Jul 05 11:53:19.988486 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.988459 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 11:53:19.988486 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.988474 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 11:53:19.990485 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.990474 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 05 11:53:19.995406 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.995392 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 05 11:53:19.995579 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.995569 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 05 11:53:19.995612 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.995603 2575 kubelet.go:486] "Attempting to sync node with API server" Jul 05 11:53:19.995654 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.995616 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 05 11:53:19.995654 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.995634 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 05 11:53:19.995654 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.995648 2575 kubelet.go:398] "Adding apiserver pod source" Jul 05 11:53:19.995759 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.995659 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 05 11:53:19.996090 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.996073 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 11:53:19.996170 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:19.996101 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 11:53:20.000360 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.000341 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 05 11:53:20.002882 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.002857 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 05 11:53:20.002953 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.002901 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 05 11:53:20.004403 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004392 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 05 11:53:20.004461 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004407 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 05 11:53:20.004461 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004413 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 05 11:53:20.004461 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004418 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 05 11:53:20.004461 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004427 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 05 11:53:20.004461 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004432 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 05 11:53:20.004461 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004439 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 05 11:53:20.004461 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004445 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 05 11:53:20.004461 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004452 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 05 11:53:20.004461 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004457 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 05 11:53:20.004461 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004465 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 05 11:53:20.004828 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.004470 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 05 11:53:20.005326 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.005311 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 05 11:53:20.005392 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.005330 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 05 11:53:20.007726 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.007707 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-207.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 05 11:53:20.007811 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.007723 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-207.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 05 11:53:20.007850 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.007841 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 05 11:53:20.009246 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.009233 2575 server.go:1268] "Started kubelet" Jul 05 11:53:20.009363 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.009310 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 05 11:53:20.009430 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.009387 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 05 11:53:20.010012 ip-10-0-142-207 systemd[1]: Started Kubernetes Kubelet. Jul 05 11:53:20.010583 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.010167 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 05 11:53:20.012125 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.012106 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 05 11:53:20.012549 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.012534 2575 server.go:310] "Adding debug handlers to kubelet server" Jul 05 11:53:20.015176 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.014008 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-207.ec2.internal.18bf629c17694dd6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-207.ec2.internal,UID:ip-10-0-142-207.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-207.ec2.internal,},FirstTimestamp:2026-07-05 11:53:20.009203158 +0000 UTC m=+0.399904950,LastTimestamp:2026-07-05 11:53:20.009203158 +0000 UTC m=+0.399904950,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-207.ec2.internal,}" Jul 05 11:53:20.015439 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.015423 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 05 11:53:20.020404 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.020385 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 05 11:53:20.020404 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.020403 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 05 11:53:20.020547 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.020493 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-207.ec2.internal\" not found" Jul 05 11:53:20.021237 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.021219 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 05 11:53:20.021310 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.021260 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 05 11:53:20.021310 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.021291 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 05 11:53:20.021310 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.021269 2575 factory.go:55] Registering systemd factory Jul 05 11:53:20.021454 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.021350 2575 factory.go:223] Registration of the systemd container factory successfully Jul 05 11:53:20.021454 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.021378 2575 reconstruct.go:97] "Volume reconstruction finished" Jul 05 11:53:20.021454 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.021386 2575 reconciler.go:29] "Reconciler: start to sync state" Jul 05 11:53:20.021784 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.021769 2575 factory.go:153] Registering CRI-O factory Jul 05 11:53:20.021784 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.021785 2575 factory.go:223] Registration of the crio container factory successfully Jul 05 11:53:20.021910 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.021831 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 05 11:53:20.021989 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.021978 2575 factory.go:103] Registering Raw factory Jul 05 11:53:20.022038 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.022016 2575 manager.go:1196] Started watching for new ooms in manager Jul 05 11:53:20.022726 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.022710 2575 manager.go:349] Starting recovery of all containers Jul 05 11:53:20.022954 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.022930 2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-207.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 05 11:53:20.023133 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.023112 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 05 11:53:20.027603 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.027578 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-22dpn" Jul 05 11:53:20.033076 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.033058 2575 manager.go:354] Recovery completed Jul 05 11:53:20.034526 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.034362 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-22dpn" Jul 05 11:53:20.035307 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.035269 2575 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 05 11:53:20.039394 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.039380 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:20.041845 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.041829 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:20.041943 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.041859 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:20.041943 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.041884 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:20.042499 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.042482 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 05 11:53:20.042499 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.042496 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 05 11:53:20.042601 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.042499 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:20.042601 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.042514 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 05 11:53:20.044645 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.044633 2575 policy_none.go:49] "None policy: Start" Jul 05 11:53:20.044694 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.044648 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 05 11:53:20.044694 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.044660 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 05 11:53:20.046365 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.046342 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:20.046437 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.046382 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:20.046437 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.046397 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:20.046667 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.046653 2575 policy_none.go:47] "Start" Jul 05 11:53:20.101490 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.084281 2575 manager.go:337] "Starting Device Plugin manager" Jul 05 11:53:20.101490 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.084328 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 05 11:53:20.101490 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.084342 2575 server.go:87] "Starting device plugin registration server" Jul 05 11:53:20.101490 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.084614 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 05 11:53:20.101490 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.084626 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 05 11:53:20.101490 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.084761 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 05 11:53:20.101490 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.084844 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 05 11:53:20.101490 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.084853 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 05 11:53:20.101490 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.086337 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 05 11:53:20.101490 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.086391 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-207.ec2.internal\" not found" Jul 05 11:53:20.121509 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.121477 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 05 11:53:20.122662 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.122641 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 05 11:53:20.122758 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.122674 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 05 11:53:20.122758 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.122695 2575 kubelet.go:2448] "Starting kubelet main sync loop" Jul 05 11:53:20.122758 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.122723 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 05 11:53:20.124828 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.124806 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:20.185633 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.185542 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:20.186523 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.186507 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:20.186601 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.186536 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:20.186601 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.186547 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:20.186601 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.186572 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.195369 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.195345 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.195369 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.195370 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-207.ec2.internal\": node \"ip-10-0-142-207.ec2.internal\" not found" Jul 05 11:53:20.217638 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.217611 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-207.ec2.internal\" not found" Jul 05 11:53:20.222833 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.222812 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-207.ec2.internal"] Jul 05 11:53:20.222918 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.222891 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:20.223739 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.223725 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:20.223802 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.223756 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:20.223802 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.223768 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:20.226208 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.226195 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:20.226309 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.226295 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.226347 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.226320 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:20.227138 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.227123 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:20.227256 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.227145 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:20.227256 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.227154 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:20.227256 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.227161 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:20.227256 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.227183 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:20.227256 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.227193 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:20.229410 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.229396 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.229464 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.229422 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:20.230115 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.230102 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:20.230196 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.230125 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:20.230196 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.230134 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:20.251494 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.251476 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-207.ec2.internal\" not found" node="ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.256216 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.256200 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-207.ec2.internal\" not found" node="ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.318748 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.318717 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-207.ec2.internal\" not found" Jul 05 11:53:20.419200 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.419163 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-207.ec2.internal\" not found" Jul 05 11:53:20.422451 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.422435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03c8ac1e9b1e3df4f5ab2b1adae689d2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal\" (UID: \"03c8ac1e9b1e3df4f5ab2b1adae689d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.422509 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.422461 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c8ac1e9b1e3df4f5ab2b1adae689d2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal\" (UID: \"03c8ac1e9b1e3df4f5ab2b1adae689d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.422509 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.422482 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3494324bd81becfa329adf53f8dc35af-config\") pod \"kube-apiserver-proxy-ip-10-0-142-207.ec2.internal\" (UID: \"3494324bd81becfa329adf53f8dc35af\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.519950 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.519867 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-207.ec2.internal\" not found" Jul 05 11:53:20.523167 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.523149 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03c8ac1e9b1e3df4f5ab2b1adae689d2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal\" (UID: \"03c8ac1e9b1e3df4f5ab2b1adae689d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.523224 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.523175 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c8ac1e9b1e3df4f5ab2b1adae689d2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal\" (UID: \"03c8ac1e9b1e3df4f5ab2b1adae689d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.523224 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.523192 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3494324bd81becfa329adf53f8dc35af-config\") pod \"kube-apiserver-proxy-ip-10-0-142-207.ec2.internal\" (UID: \"3494324bd81becfa329adf53f8dc35af\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.523224 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.523218 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3494324bd81becfa329adf53f8dc35af-config\") pod \"kube-apiserver-proxy-ip-10-0-142-207.ec2.internal\" (UID: \"3494324bd81becfa329adf53f8dc35af\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.523316 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.523234 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03c8ac1e9b1e3df4f5ab2b1adae689d2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal\" (UID: \"03c8ac1e9b1e3df4f5ab2b1adae689d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.523316 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.523242 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c8ac1e9b1e3df4f5ab2b1adae689d2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal\" (UID: \"03c8ac1e9b1e3df4f5ab2b1adae689d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.555313 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.555284 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.559024 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.559007 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.620976 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.620942 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-207.ec2.internal\" not found" Jul 05 11:53:20.722869 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.722809 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-207.ec2.internal\" not found" Jul 05 11:53:20.823850 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:20.823820 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-207.ec2.internal\" not found" Jul 05 11:53:20.910989 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.910961 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:20.921500 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.921475 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.931057 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.931033 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 11:53:20.932534 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.932519 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" Jul 05 11:53:20.944291 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.944268 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 11:53:20.952044 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.952018 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 05 11:53:20.952206 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.952183 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 11:53:20.952273 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.952212 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 11:53:20.997785 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:20.997727 2575 apiserver.go:52] "Watching apiserver" Jul 05 11:53:21.015041 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.015005 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 05 11:53:21.016920 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.016872 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-dhzh9","openshift-network-diagnostics/network-check-target-rt4k7","kube-system/global-pull-secret-syncer-8z5fg","kube-system/konnectivity-agent-j6nrt","kube-system/kube-apiserver-proxy-ip-10-0-142-207.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj","openshift-image-registry/node-ca-s2dpw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal","openshift-network-operator/iptables-alerter-rmzcs","openshift-ovn-kubernetes/ovnkube-node-ngxcf","openshift-cluster-node-tuning-operator/tuned-9mt88","openshift-dns/node-resolver-hb68f","openshift-multus/multus-additional-cni-plugins-6bc4j","openshift-multus/multus-gjqwd"] Jul 05 11:53:21.019784 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.019761 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.020933 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.020911 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 05 11:53:21.022507 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.022479 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 05 11:53:21.023735 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.022853 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 05 11:53:21.023735 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.022872 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 05 11:53:21.023735 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.023017 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c9qhj\"" Jul 05 11:53:21.024012 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.023839 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:21.024012 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.023919 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:21.024280 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.024259 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 05 11:53:21.024280 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.024268 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 05 11:53:21.024437 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.024319 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 05 11:53:21.025220 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025199 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-var-lib-openvswitch\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025321 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025233 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-run-ovn\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025321 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025258 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-cni-netd\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025321 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025472 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025329 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-ovnkube-config\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025472 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025370 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-env-overrides\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025472 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025426 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-ovn-node-metrics-cert\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025577 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025471 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-kubelet\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025577 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025502 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-run-openvswitch\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025577 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025518 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-node-log\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025577 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025537 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-ovnkube-script-lib\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025577 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025559 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjz24\" (UniqueName: \"kubernetes.io/projected/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-kube-api-access-vjz24\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025577 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025574 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pwx6\" (UniqueName: \"kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6\") pod \"network-check-target-rt4k7\" (UID: \"3997821f-4556-407b-9b73-daf87e87f9b1\") " pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:21.025828 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025590 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-systemd-units\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025828 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-slash\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025828 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025636 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-run-netns\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025828 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025671 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-etc-openvswitch\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025828 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025705 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-log-socket\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025828 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025748 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-run-ovn-kubernetes\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025828 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025769 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-cni-bin\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.025828 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.025793 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-run-systemd\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.026239 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.026049 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hb68f" Jul 05 11:53:21.028466 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.028447 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.028570 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.028530 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 05 11:53:21.028897 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.028881 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nl7dm\"" Jul 05 11:53:21.029090 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.029075 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 05 11:53:21.030968 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.030942 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:21.031350 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.031334 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 05 11:53:21.031492 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.031476 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jql6j\"" Jul 05 11:53:21.031567 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.031548 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 05 11:53:21.033108 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.032981 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 11:53:21.033275 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.033257 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 05 11:53:21.033479 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.033457 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 05 11:53:21.033603 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.033543 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.033678 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.033667 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6c8mp\"" Jul 05 11:53:21.035875 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.035857 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s2dpw" Jul 05 11:53:21.036317 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.036295 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 05 11:53:21.036413 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.036323 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 05 11:53:21.036413 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.036323 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t6sz5\"" Jul 05 11:53:21.036413 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.036299 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 05 11:53:21.038187 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.038160 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-04 11:48:20 +0000 UTC" deadline="2028-03-14 17:42:03.214333706 +0000 UTC" Jul 05 11:53:21.038235 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.038189 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14837h48m42.176148158s" Jul 05 11:53:21.038270 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.038258 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rmzcs" Jul 05 11:53:21.038650 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.038634 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 05 11:53:21.038725 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.038704 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jfpp7\"" Jul 05 11:53:21.038831 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.038815 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 05 11:53:21.038908 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.038843 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 05 11:53:21.040701 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.040681 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:21.040790 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.040748 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:21.040842 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.040824 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 05 11:53:21.041088 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.041057 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t6ct2\"" Jul 05 11:53:21.041088 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.041075 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 05 11:53:21.041211 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.041127 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 05 11:53:21.043268 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.043248 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:21.043373 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.043314 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:21.045881 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.045861 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.048310 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.048289 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.048546 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.048526 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 05 11:53:21.048546 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.048538 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 05 11:53:21.048669 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.048616 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 05 11:53:21.048705 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.048693 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 05 11:53:21.048773 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.048760 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g5qcq\"" Jul 05 11:53:21.049376 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.049359 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 05 11:53:21.051695 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.051677 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 05 11:53:21.051775 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.051680 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rh2dl\"" Jul 05 11:53:21.063515 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.063495 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t52cd" Jul 05 11:53:21.071608 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.071578 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t52cd" Jul 05 11:53:21.122857 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.122833 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 05 11:53:21.126028 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126007 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-sysctl-conf\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.126028 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126032 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-var-lib-kubelet\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.126216 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126056 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-var-lib-openvswitch\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.126216 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126101 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-run-ovn\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.126216 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126164 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-var-lib-openvswitch\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.126216 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126168 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-run-ovn\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.126216 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126186 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-socket-dir-parent\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.126433 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126226 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-run-netns\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.126433 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126254 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkxmn\" (UniqueName: \"kubernetes.io/projected/9163d99b-81f1-49d9-85c4-e35200fb1b1c-kube-api-access-hkxmn\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.126433 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126281 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-systemd\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.126433 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126303 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8fm2\" (UniqueName: \"kubernetes.io/projected/afa4b888-1bc8-4865-911e-5720c5b0028a-kube-api-access-s8fm2\") pod \"node-ca-s2dpw\" (UID: \"afa4b888-1bc8-4865-911e-5720c5b0028a\") " pod="openshift-image-registry/node-ca-s2dpw" Jul 05 11:53:21.126433 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126349 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/746ca75a-b87d-4153-9e6a-74845453758c-dbus\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:21.126433 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126375 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43726046-5603-4270-94d8-4e3f27f984b7-cni-binary-copy\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.126433 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126398 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-modprobe-d\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.126433 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126422 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-sysconfig\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.126703 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126448 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xb2l\" (UniqueName: \"kubernetes.io/projected/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-kube-api-access-9xb2l\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.126703 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126476 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-ovnkube-script-lib\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.126703 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126503 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pwx6\" (UniqueName: \"kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6\") pod \"network-check-target-rt4k7\" (UID: \"3997821f-4556-407b-9b73-daf87e87f9b1\") " pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:21.126703 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126646 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-kubernetes\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.126703 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126684 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/746ca75a-b87d-4153-9e6a-74845453758c-kubelet-config\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:21.126870 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126726 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-systemd-units\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.126870 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126776 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-systemd-units\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.126870 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126805 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-slash\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.126870 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126831 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-slash\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.126870 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126842 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-log-socket\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126867 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-cni-dir\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126907 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-cnibin\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126921 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-log-socket\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126933 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-var-lib-cni-multus\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126957 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/161771f1-5ef9-49fd-924f-a9a89873ebce-tmp\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.126982 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ff7494e-12c6-42da-af77-38131222c9bf-iptables-alerter-script\") pod \"iptables-alerter-rmzcs\" (UID: \"4ff7494e-12c6-42da-af77-38131222c9bf\") " pod="openshift-network-operator/iptables-alerter-rmzcs" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127019 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-os-release\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127045 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-var-lib-kubelet\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127071 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-registration-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127095 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afa4b888-1bc8-4865-911e-5720c5b0028a-serviceca\") pod \"node-ca-s2dpw\" (UID: \"afa4b888-1bc8-4865-911e-5720c5b0028a\") " pod="openshift-image-registry/node-ca-s2dpw" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127120 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-system-cni-dir\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.127155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127147 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-os-release\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127170 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-run\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127176 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-ovnkube-script-lib\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127194 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afa4b888-1bc8-4865-911e-5720c5b0028a-host\") pod \"node-ca-s2dpw\" (UID: \"afa4b888-1bc8-4865-911e-5720c5b0028a\") " pod="openshift-image-registry/node-ca-s2dpw" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127218 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxgds\" (UniqueName: \"kubernetes.io/projected/43726046-5603-4270-94d8-4e3f27f984b7-kube-api-access-lxgds\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127242 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9163d99b-81f1-49d9-85c4-e35200fb1b1c-cni-binary-copy\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127263 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9930e6e9-d5bc-4384-a9de-97740b2b692d-hosts-file\") pod \"node-resolver-hb68f\" (UID: \"9930e6e9-d5bc-4384-a9de-97740b2b692d\") " pod="openshift-dns/node-resolver-hb68f" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127281 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-run-netns\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127298 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-cni-bin\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127336 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-run-netns\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127338 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-cni-bin\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127352 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-cni-netd\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127379 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-system-cni-dir\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127402 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-var-lib-cni-bin\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127418 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-sysctl-d\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127432 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-etc-selinux\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127384 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-cni-netd\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.127747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127471 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-run-systemd\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127498 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127510 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-run-systemd\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127523 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-run-k8s-cni-cncf-io\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127547 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-conf-dir\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127567 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-daemon-config\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-sys\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127656 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ff7494e-12c6-42da-af77-38131222c9bf-host-slash\") pod \"iptables-alerter-rmzcs\" (UID: \"4ff7494e-12c6-42da-af77-38131222c9bf\") " pod="openshift-network-operator/iptables-alerter-rmzcs" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127685 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbxz4\" (UniqueName: \"kubernetes.io/projected/4ff7494e-12c6-42da-af77-38131222c9bf-kube-api-access-qbxz4\") pod \"iptables-alerter-rmzcs\" (UID: \"4ff7494e-12c6-42da-af77-38131222c9bf\") " pod="openshift-network-operator/iptables-alerter-rmzcs" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127713 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-kubelet\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127747 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-node-log\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127773 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-etc-kubernetes\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127778 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-kubelet\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127795 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-node-log\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127868 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-socket-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127907 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpbdm\" (UniqueName: \"kubernetes.io/projected/9930e6e9-d5bc-4384-a9de-97740b2b692d-kube-api-access-vpbdm\") pod \"node-resolver-hb68f\" (UID: \"9930e6e9-d5bc-4384-a9de-97740b2b692d\") " pod="openshift-dns/node-resolver-hb68f" Jul 05 11:53:21.128429 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127936 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d473ee62-6227-440a-bad0-ef141bb75628-agent-certs\") pod \"konnectivity-agent-j6nrt\" (UID: \"d473ee62-6227-440a-bad0-ef141bb75628\") " pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127961 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-lib-modules\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127978 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfxmt\" (UniqueName: \"kubernetes.io/projected/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-kube-api-access-mfxmt\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.127997 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-etc-openvswitch\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128019 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128040 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-etc-openvswitch\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128078 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/43726046-5603-4270-94d8-4e3f27f984b7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128094 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-host\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128119 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128140 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j54jr\" (UniqueName: \"kubernetes.io/projected/161771f1-5ef9-49fd-924f-a9a89873ebce-kube-api-access-j54jr\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128157 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d473ee62-6227-440a-bad0-ef141bb75628-konnectivity-ca\") pod \"konnectivity-agent-j6nrt\" (UID: \"d473ee62-6227-440a-bad0-ef141bb75628\") " pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-ovnkube-config\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128189 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-cnibin\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-hostroot\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128218 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-device-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128233 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-run-openvswitch\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129039 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128250 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128274 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9930e6e9-d5bc-4384-a9de-97740b2b692d-tmp-dir\") pod \"node-resolver-hb68f\" (UID: \"9930e6e9-d5bc-4384-a9de-97740b2b692d\") " pod="openshift-dns/node-resolver-hb68f" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128293 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-run-openvswitch\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128298 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-tuned\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128334 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-sys-fs\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128352 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128393 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-run-ovn-kubernetes\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128444 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-host-run-ovn-kubernetes\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128450 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-env-overrides\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128488 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-ovn-node-metrics-cert\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128520 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjz24\" (UniqueName: \"kubernetes.io/projected/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-kube-api-access-vjz24\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128548 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/43726046-5603-4270-94d8-4e3f27f984b7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128575 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-run-multus-certs\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128662 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-ovnkube-config\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128775 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 05 11:53:21.129747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.128836 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-env-overrides\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.131913 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.131892 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-ovn-node-metrics-cert\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.138320 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.138295 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:21.138320 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.138320 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:21.138443 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.138331 2575 projected.go:196] Error preparing data for projected volume kube-api-access-2pwx6 for pod openshift-network-diagnostics/network-check-target-rt4k7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:21.138443 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.138387 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6 podName:3997821f-4556-407b-9b73-daf87e87f9b1 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:21.638370231 +0000 UTC m=+2.029071995 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2pwx6" (UniqueName: "kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6") pod "network-check-target-rt4k7" (UID: "3997821f-4556-407b-9b73-daf87e87f9b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:21.141102 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.141084 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjz24\" (UniqueName: \"kubernetes.io/projected/447db484-5c4b-4e4e-86f7-acbcb5c4f9cf-kube-api-access-vjz24\") pod \"ovnkube-node-ngxcf\" (UID: \"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.228836 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.228796 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-system-cni-dir\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.228836 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.228842 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-var-lib-cni-bin\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.228859 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-sysctl-d\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.228884 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-etc-selinux\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.228892 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-system-cni-dir\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.228968 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-var-lib-cni-bin\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.228983 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229027 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-run-k8s-cni-cncf-io\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229040 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-etc-selinux\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-conf-dir\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229043 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-sysctl-d\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229083 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-run-k8s-cni-cncf-io\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229116 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-conf-dir\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.229155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229158 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229172 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-daemon-config\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229198 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-sys\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229246 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-sys\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229295 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ff7494e-12c6-42da-af77-38131222c9bf-host-slash\") pod \"iptables-alerter-rmzcs\" (UID: \"4ff7494e-12c6-42da-af77-38131222c9bf\") " pod="openshift-network-operator/iptables-alerter-rmzcs" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229320 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbxz4\" (UniqueName: \"kubernetes.io/projected/4ff7494e-12c6-42da-af77-38131222c9bf-kube-api-access-qbxz4\") pod \"iptables-alerter-rmzcs\" (UID: \"4ff7494e-12c6-42da-af77-38131222c9bf\") " pod="openshift-network-operator/iptables-alerter-rmzcs" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229338 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-etc-kubernetes\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229358 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229383 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-socket-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229381 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-etc-kubernetes\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229421 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ff7494e-12c6-42da-af77-38131222c9bf-host-slash\") pod \"iptables-alerter-rmzcs\" (UID: \"4ff7494e-12c6-42da-af77-38131222c9bf\") " pod="openshift-network-operator/iptables-alerter-rmzcs" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229412 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpbdm\" (UniqueName: \"kubernetes.io/projected/9930e6e9-d5bc-4384-a9de-97740b2b692d-kube-api-access-vpbdm\") pod \"node-resolver-hb68f\" (UID: \"9930e6e9-d5bc-4384-a9de-97740b2b692d\") " pod="openshift-dns/node-resolver-hb68f" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229466 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229497 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d473ee62-6227-440a-bad0-ef141bb75628-agent-certs\") pod \"konnectivity-agent-j6nrt\" (UID: \"d473ee62-6227-440a-bad0-ef141bb75628\") " pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229518 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-lib-modules\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229528 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-socket-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229626 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-lib-modules\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.229635 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229616 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfxmt\" (UniqueName: \"kubernetes.io/projected/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-kube-api-access-mfxmt\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229701 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/43726046-5603-4270-94d8-4e3f27f984b7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229782 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-host\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229881 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j54jr\" (UniqueName: \"kubernetes.io/projected/161771f1-5ef9-49fd-924f-a9a89873ebce-kube-api-access-j54jr\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229944 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-host\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229951 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d473ee62-6227-440a-bad0-ef141bb75628-konnectivity-ca\") pod \"konnectivity-agent-j6nrt\" (UID: \"d473ee62-6227-440a-bad0-ef141bb75628\") " pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.229997 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-cnibin\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230025 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-hostroot\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230118 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-device-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230151 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230177 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9930e6e9-d5bc-4384-a9de-97740b2b692d-tmp-dir\") pod \"node-resolver-hb68f\" (UID: \"9930e6e9-d5bc-4384-a9de-97740b2b692d\") " pod="openshift-dns/node-resolver-hb68f" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230079 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-cnibin\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230194 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-hostroot\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230231 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-tuned\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230273 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-sys-fs\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230284 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-device-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230325 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:21.230469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230358 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/43726046-5603-4270-94d8-4e3f27f984b7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230438 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-run-multus-certs\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230462 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-sysctl-conf\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230480 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-daemon-config\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230553 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-sys-fs\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230486 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-var-lib-kubelet\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230616 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-socket-dir-parent\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.230638 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230572 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-var-lib-kubelet\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.230668 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230684 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-run-netns\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.230716 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs podName:793d5b93-3b3a-4f7b-99fb-38db6730bfa6 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:21.730695297 +0000 UTC m=+2.121397054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs") pod "network-metrics-daemon-dhzh9" (UID: "793d5b93-3b3a-4f7b-99fb-38db6730bfa6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230758 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-run-multus-certs\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230829 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-socket-dir-parent\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230863 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/43726046-5603-4270-94d8-4e3f27f984b7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230916 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-run-netns\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230915 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d473ee62-6227-440a-bad0-ef141bb75628-konnectivity-ca\") pod \"konnectivity-agent-j6nrt\" (UID: \"d473ee62-6227-440a-bad0-ef141bb75628\") " pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:21.231329 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.230914 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-sysctl-conf\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.230984 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret podName:746ca75a-b87d-4153-9e6a-74845453758c nodeName:}" failed. No retries permitted until 2026-07-05 11:53:21.730943859 +0000 UTC m=+2.121645608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret") pod "global-pull-secret-syncer-8z5fg" (UID: "746ca75a-b87d-4153-9e6a-74845453758c") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231017 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkxmn\" (UniqueName: \"kubernetes.io/projected/9163d99b-81f1-49d9-85c4-e35200fb1b1c-kube-api-access-hkxmn\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231047 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-systemd\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231075 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8fm2\" (UniqueName: \"kubernetes.io/projected/afa4b888-1bc8-4865-911e-5720c5b0028a-kube-api-access-s8fm2\") pod \"node-ca-s2dpw\" (UID: \"afa4b888-1bc8-4865-911e-5720c5b0028a\") " pod="openshift-image-registry/node-ca-s2dpw" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231116 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/746ca75a-b87d-4153-9e6a-74845453758c-dbus\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231155 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/43726046-5603-4270-94d8-4e3f27f984b7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43726046-5603-4270-94d8-4e3f27f984b7-cni-binary-copy\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231191 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9930e6e9-d5bc-4384-a9de-97740b2b692d-tmp-dir\") pod \"node-resolver-hb68f\" (UID: \"9930e6e9-d5bc-4384-a9de-97740b2b692d\") " pod="openshift-dns/node-resolver-hb68f" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231209 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-modprobe-d\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231242 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-sysconfig\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231249 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-systemd\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231268 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xb2l\" (UniqueName: \"kubernetes.io/projected/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-kube-api-access-9xb2l\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231331 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-sysconfig\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231355 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-modprobe-d\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231398 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-kubernetes\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.232302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231479 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/746ca75a-b87d-4153-9e6a-74845453758c-kubelet-config\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231508 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-kubernetes\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231541 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/746ca75a-b87d-4153-9e6a-74845453758c-dbus\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231549 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-cni-dir\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231578 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-cnibin\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231592 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/746ca75a-b87d-4153-9e6a-74845453758c-kubelet-config\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231605 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-var-lib-cni-multus\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231614 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-multus-cni-dir\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231642 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-cnibin\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231631 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/161771f1-5ef9-49fd-924f-a9a89873ebce-tmp\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231683 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-var-lib-cni-multus\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231687 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ff7494e-12c6-42da-af77-38131222c9bf-iptables-alerter-script\") pod \"iptables-alerter-rmzcs\" (UID: \"4ff7494e-12c6-42da-af77-38131222c9bf\") " pod="openshift-network-operator/iptables-alerter-rmzcs" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231726 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-os-release\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231751 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-var-lib-kubelet\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231775 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-registration-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231814 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43726046-5603-4270-94d8-4e3f27f984b7-cni-binary-copy\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231831 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-os-release\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231847 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afa4b888-1bc8-4865-911e-5720c5b0028a-serviceca\") pod \"node-ca-s2dpw\" (UID: \"afa4b888-1bc8-4865-911e-5720c5b0028a\") " pod="openshift-image-registry/node-ca-s2dpw" Jul 05 11:53:21.233070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231899 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-host-var-lib-kubelet\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231917 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-registration-dir\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231951 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-system-cni-dir\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.231981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-os-release\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232004 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-run\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232020 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43726046-5603-4270-94d8-4e3f27f984b7-system-cni-dir\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232025 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afa4b888-1bc8-4865-911e-5720c5b0028a-host\") pod \"node-ca-s2dpw\" (UID: \"afa4b888-1bc8-4865-911e-5720c5b0028a\") " pod="openshift-image-registry/node-ca-s2dpw" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232055 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afa4b888-1bc8-4865-911e-5720c5b0028a-host\") pod \"node-ca-s2dpw\" (UID: \"afa4b888-1bc8-4865-911e-5720c5b0028a\") " pod="openshift-image-registry/node-ca-s2dpw" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232070 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxgds\" (UniqueName: \"kubernetes.io/projected/43726046-5603-4270-94d8-4e3f27f984b7-kube-api-access-lxgds\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232112 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9163d99b-81f1-49d9-85c4-e35200fb1b1c-os-release\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232122 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9163d99b-81f1-49d9-85c4-e35200fb1b1c-cni-binary-copy\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232153 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/161771f1-5ef9-49fd-924f-a9a89873ebce-run\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232162 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9930e6e9-d5bc-4384-a9de-97740b2b692d-hosts-file\") pod \"node-resolver-hb68f\" (UID: \"9930e6e9-d5bc-4384-a9de-97740b2b692d\") " pod="openshift-dns/node-resolver-hb68f" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232275 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9930e6e9-d5bc-4384-a9de-97740b2b692d-hosts-file\") pod \"node-resolver-hb68f\" (UID: \"9930e6e9-d5bc-4384-a9de-97740b2b692d\") " pod="openshift-dns/node-resolver-hb68f" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232293 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ff7494e-12c6-42da-af77-38131222c9bf-iptables-alerter-script\") pod \"iptables-alerter-rmzcs\" (UID: \"4ff7494e-12c6-42da-af77-38131222c9bf\") " pod="openshift-network-operator/iptables-alerter-rmzcs" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.232443 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afa4b888-1bc8-4865-911e-5720c5b0028a-serviceca\") pod \"node-ca-s2dpw\" (UID: \"afa4b888-1bc8-4865-911e-5720c5b0028a\") " pod="openshift-image-registry/node-ca-s2dpw" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.233182 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d473ee62-6227-440a-bad0-ef141bb75628-agent-certs\") pod \"konnectivity-agent-j6nrt\" (UID: \"d473ee62-6227-440a-bad0-ef141bb75628\") " pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:21.233915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.233463 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9163d99b-81f1-49d9-85c4-e35200fb1b1c-cni-binary-copy\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.236191 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.236168 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/161771f1-5ef9-49fd-924f-a9a89873ebce-tmp\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.236346 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.236327 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/161771f1-5ef9-49fd-924f-a9a89873ebce-etc-tuned\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.239611 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.239589 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j54jr\" (UniqueName: \"kubernetes.io/projected/161771f1-5ef9-49fd-924f-a9a89873ebce-kube-api-access-j54jr\") pod \"tuned-9mt88\" (UID: \"161771f1-5ef9-49fd-924f-a9a89873ebce\") " pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.239778 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.239754 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpbdm\" (UniqueName: \"kubernetes.io/projected/9930e6e9-d5bc-4384-a9de-97740b2b692d-kube-api-access-vpbdm\") pod \"node-resolver-hb68f\" (UID: \"9930e6e9-d5bc-4384-a9de-97740b2b692d\") " pod="openshift-dns/node-resolver-hb68f" Jul 05 11:53:21.239906 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.239883 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbxz4\" (UniqueName: \"kubernetes.io/projected/4ff7494e-12c6-42da-af77-38131222c9bf-kube-api-access-qbxz4\") pod \"iptables-alerter-rmzcs\" (UID: \"4ff7494e-12c6-42da-af77-38131222c9bf\") " pod="openshift-network-operator/iptables-alerter-rmzcs" Jul 05 11:53:21.239975 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.239920 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfxmt\" (UniqueName: \"kubernetes.io/projected/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-kube-api-access-mfxmt\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:21.244295 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.244269 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkxmn\" (UniqueName: \"kubernetes.io/projected/9163d99b-81f1-49d9-85c4-e35200fb1b1c-kube-api-access-hkxmn\") pod \"multus-gjqwd\" (UID: \"9163d99b-81f1-49d9-85c4-e35200fb1b1c\") " pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.245130 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.245103 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8fm2\" (UniqueName: \"kubernetes.io/projected/afa4b888-1bc8-4865-911e-5720c5b0028a-kube-api-access-s8fm2\") pod \"node-ca-s2dpw\" (UID: \"afa4b888-1bc8-4865-911e-5720c5b0028a\") " pod="openshift-image-registry/node-ca-s2dpw" Jul 05 11:53:21.245216 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.245205 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xb2l\" (UniqueName: \"kubernetes.io/projected/82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa-kube-api-access-9xb2l\") pod \"aws-ebs-csi-driver-node-l7xgj\" (UID: \"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.245277 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.245259 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxgds\" (UniqueName: \"kubernetes.io/projected/43726046-5603-4270-94d8-4e3f27f984b7-kube-api-access-lxgds\") pod \"multus-additional-cni-plugins-6bc4j\" (UID: \"43726046-5603-4270-94d8-4e3f27f984b7\") " pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.308130 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:21.308091 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3494324bd81becfa329adf53f8dc35af.slice/crio-664a71e8ede0875d6db76369f88cef2cf51c4eede698121b243952dff5c24167 WatchSource:0}: Error finding container 664a71e8ede0875d6db76369f88cef2cf51c4eede698121b243952dff5c24167: Status 404 returned error can't find the container with id 664a71e8ede0875d6db76369f88cef2cf51c4eede698121b243952dff5c24167 Jul 05 11:53:21.309168 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:21.309141 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c8ac1e9b1e3df4f5ab2b1adae689d2.slice/crio-d0170c58d79d4830a1c1bb3c560e48e96d72bf7e6022c7d1ff0b5ec26fa41b60 WatchSource:0}: Error finding container d0170c58d79d4830a1c1bb3c560e48e96d72bf7e6022c7d1ff0b5ec26fa41b60: Status 404 returned error can't find the container with id d0170c58d79d4830a1c1bb3c560e48e96d72bf7e6022c7d1ff0b5ec26fa41b60 Jul 05 11:53:21.315242 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.315220 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 11:53:21.344778 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.344690 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:21.347298 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.347279 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:21.355027 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:21.355003 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod447db484_5c4b_4e4e_86f7_acbcb5c4f9cf.slice/crio-ef663fdaa83e280ceb7c804199dbece4170370e2fc9135e5c6a1abe9b0f4b2fc WatchSource:0}: Error finding container ef663fdaa83e280ceb7c804199dbece4170370e2fc9135e5c6a1abe9b0f4b2fc: Status 404 returned error can't find the container with id ef663fdaa83e280ceb7c804199dbece4170370e2fc9135e5c6a1abe9b0f4b2fc Jul 05 11:53:21.364810 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.364788 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hb68f" Jul 05 11:53:21.372101 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.372079 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9mt88" Jul 05 11:53:21.373558 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:21.373536 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9930e6e9_d5bc_4384_a9de_97740b2b692d.slice/crio-fdd4ce6ae68849311afa19bc10035be3b89b81e3e31013a63ab37af7ee9a6ceb WatchSource:0}: Error finding container fdd4ce6ae68849311afa19bc10035be3b89b81e3e31013a63ab37af7ee9a6ceb: Status 404 returned error can't find the container with id fdd4ce6ae68849311afa19bc10035be3b89b81e3e31013a63ab37af7ee9a6ceb Jul 05 11:53:21.381637 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:21.381613 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161771f1_5ef9_49fd_924f_a9a89873ebce.slice/crio-ebc0b97001091154c49ea8fd9a4568c723f8d4e3c60db4a8cd2553e77c4c2f68 WatchSource:0}: Error finding container ebc0b97001091154c49ea8fd9a4568c723f8d4e3c60db4a8cd2553e77c4c2f68: Status 404 returned error can't find the container with id ebc0b97001091154c49ea8fd9a4568c723f8d4e3c60db4a8cd2553e77c4c2f68 Jul 05 11:53:21.385731 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.385713 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:21.394973 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:21.394930 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd473ee62_6227_440a_bad0_ef141bb75628.slice/crio-0168b70fba1f816ba9f66abd6560937ed079c9328e83ecdc6cd0b09cd694fe8c WatchSource:0}: Error finding container 0168b70fba1f816ba9f66abd6560937ed079c9328e83ecdc6cd0b09cd694fe8c: Status 404 returned error can't find the container with id 0168b70fba1f816ba9f66abd6560937ed079c9328e83ecdc6cd0b09cd694fe8c Jul 05 11:53:21.401948 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.401924 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" Jul 05 11:53:21.410559 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:21.410536 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82fdc3d7_4fd4_40ef_8c51_4b18c36aa2aa.slice/crio-392641b9d63861855d09fdb6d6d86408fd95a689a36f362389ce3b3c4312ae70 WatchSource:0}: Error finding container 392641b9d63861855d09fdb6d6d86408fd95a689a36f362389ce3b3c4312ae70: Status 404 returned error can't find the container with id 392641b9d63861855d09fdb6d6d86408fd95a689a36f362389ce3b3c4312ae70 Jul 05 11:53:21.417940 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.417921 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s2dpw" Jul 05 11:53:21.423001 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.422983 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rmzcs" Jul 05 11:53:21.427732 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:21.427708 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa4b888_1bc8_4865_911e_5720c5b0028a.slice/crio-b37c12a427937ce2b28c722795c3322dbf0efe69d45b46feb2f221e2a5655792 WatchSource:0}: Error finding container b37c12a427937ce2b28c722795c3322dbf0efe69d45b46feb2f221e2a5655792: Status 404 returned error can't find the container with id b37c12a427937ce2b28c722795c3322dbf0efe69d45b46feb2f221e2a5655792 Jul 05 11:53:21.431585 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:21.431564 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff7494e_12c6_42da_af77_38131222c9bf.slice/crio-6f4c3db612c27c315fa4d238c9515f8c36fc2d6f810ff97d9e530f31dff9ebf6 WatchSource:0}: Error finding container 6f4c3db612c27c315fa4d238c9515f8c36fc2d6f810ff97d9e530f31dff9ebf6: Status 404 returned error can't find the container with id 6f4c3db612c27c315fa4d238c9515f8c36fc2d6f810ff97d9e530f31dff9ebf6 Jul 05 11:53:21.456266 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.456227 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6bc4j" Jul 05 11:53:21.461522 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.461499 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gjqwd" Jul 05 11:53:21.469806 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.469782 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:21.470541 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:21.470521 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43726046_5603_4270_94d8_4e3f27f984b7.slice/crio-bba2622a47fd120ad5ae29e3217943485b1d6af13f131249d1aba84736e7b59e WatchSource:0}: Error finding container bba2622a47fd120ad5ae29e3217943485b1d6af13f131249d1aba84736e7b59e: Status 404 returned error can't find the container with id bba2622a47fd120ad5ae29e3217943485b1d6af13f131249d1aba84736e7b59e Jul 05 11:53:21.471117 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:21.471094 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9163d99b_81f1_49d9_85c4_e35200fb1b1c.slice/crio-febb201d06cbe3f9da2b917ef689a2bfee1d78c19f5be4fb917961a7d1839c32 WatchSource:0}: Error finding container febb201d06cbe3f9da2b917ef689a2bfee1d78c19f5be4fb917961a7d1839c32: Status 404 returned error can't find the container with id febb201d06cbe3f9da2b917ef689a2bfee1d78c19f5be4fb917961a7d1839c32 Jul 05 11:53:21.472798 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.472777 2575 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-gjqwd_openshift-multus(9163d99b-81f1-49d9-85c4-e35200fb1b1c): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 05 11:53:21.472879 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.472817 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-gjqwd" podUID="9163d99b-81f1-49d9-85c4-e35200fb1b1c" Jul 05 11:53:21.735057 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.734962 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:21.735057 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.735005 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:21.735267 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.735110 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pwx6\" (UniqueName: \"kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6\") pod \"network-check-target-rt4k7\" (UID: \"3997821f-4556-407b-9b73-daf87e87f9b1\") " pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:21.735267 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.735116 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:21.735267 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.735181 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs podName:793d5b93-3b3a-4f7b-99fb-38db6730bfa6 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:22.735164228 +0000 UTC m=+3.125865968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs") pod "network-metrics-daemon-dhzh9" (UID: "793d5b93-3b3a-4f7b-99fb-38db6730bfa6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:21.735267 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.735216 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:21.735474 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.735277 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret podName:746ca75a-b87d-4153-9e6a-74845453758c nodeName:}" failed. No retries permitted until 2026-07-05 11:53:22.735261936 +0000 UTC m=+3.125963701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret") pod "global-pull-secret-syncer-8z5fg" (UID: "746ca75a-b87d-4153-9e6a-74845453758c") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:21.735474 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.735221 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:21.735474 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.735317 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:21.735474 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.735331 2575 projected.go:196] Error preparing data for projected volume kube-api-access-2pwx6 for pod openshift-network-diagnostics/network-check-target-rt4k7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:21.735474 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:21.735379 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6 podName:3997821f-4556-407b-9b73-daf87e87f9b1 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:22.735363403 +0000 UTC m=+3.126065160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pwx6" (UniqueName: "kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6") pod "network-check-target-rt4k7" (UID: "3997821f-4556-407b-9b73-daf87e87f9b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:21.953750 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:21.953716 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:22.074738 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.074691 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 11:48:21 +0000 UTC" deadline="2028-04-09 04:51:08.85993953 +0000 UTC" Jul 05 11:53:22.074738 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.074736 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15448h57m46.785208673s" Jul 05 11:53:22.123640 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.123597 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:22.123813 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:22.123763 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:22.177574 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.177402 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bc4j" event={"ID":"43726046-5603-4270-94d8-4e3f27f984b7","Type":"ContainerStarted","Data":"bba2622a47fd120ad5ae29e3217943485b1d6af13f131249d1aba84736e7b59e"} Jul 05 11:53:22.188496 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.188387 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rmzcs" event={"ID":"4ff7494e-12c6-42da-af77-38131222c9bf","Type":"ContainerStarted","Data":"6f4c3db612c27c315fa4d238c9515f8c36fc2d6f810ff97d9e530f31dff9ebf6"} Jul 05 11:53:22.189958 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.189881 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" event={"ID":"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa","Type":"ContainerStarted","Data":"392641b9d63861855d09fdb6d6d86408fd95a689a36f362389ce3b3c4312ae70"} Jul 05 11:53:22.195454 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.195410 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9mt88" event={"ID":"161771f1-5ef9-49fd-924f-a9a89873ebce","Type":"ContainerStarted","Data":"ebc0b97001091154c49ea8fd9a4568c723f8d4e3c60db4a8cd2553e77c4c2f68"} Jul 05 11:53:22.211506 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.211473 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-207.ec2.internal" event={"ID":"3494324bd81becfa329adf53f8dc35af","Type":"ContainerStarted","Data":"664a71e8ede0875d6db76369f88cef2cf51c4eede698121b243952dff5c24167"} Jul 05 11:53:22.221320 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.221237 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" event={"ID":"03c8ac1e9b1e3df4f5ab2b1adae689d2","Type":"ContainerStarted","Data":"d0170c58d79d4830a1c1bb3c560e48e96d72bf7e6022c7d1ff0b5ec26fa41b60"} Jul 05 11:53:22.231977 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.231035 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gjqwd" event={"ID":"9163d99b-81f1-49d9-85c4-e35200fb1b1c","Type":"ContainerStarted","Data":"febb201d06cbe3f9da2b917ef689a2bfee1d78c19f5be4fb917961a7d1839c32"} Jul 05 11:53:22.237900 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:22.237619 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55ec7147f859301bbb8288966c8a21c5db729200f3c3d91db671978baef79517\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-gjqwd" podUID="9163d99b-81f1-49d9-85c4-e35200fb1b1c" Jul 05 11:53:22.240314 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.240284 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s2dpw" event={"ID":"afa4b888-1bc8-4865-911e-5720c5b0028a","Type":"ContainerStarted","Data":"b37c12a427937ce2b28c722795c3322dbf0efe69d45b46feb2f221e2a5655792"} Jul 05 11:53:22.273736 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.273697 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j6nrt" event={"ID":"d473ee62-6227-440a-bad0-ef141bb75628","Type":"ContainerStarted","Data":"0168b70fba1f816ba9f66abd6560937ed079c9328e83ecdc6cd0b09cd694fe8c"} Jul 05 11:53:22.287682 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.287643 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hb68f" event={"ID":"9930e6e9-d5bc-4384-a9de-97740b2b692d","Type":"ContainerStarted","Data":"fdd4ce6ae68849311afa19bc10035be3b89b81e3e31013a63ab37af7ee9a6ceb"} Jul 05 11:53:22.298177 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.298142 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" event={"ID":"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf","Type":"ContainerStarted","Data":"ef663fdaa83e280ceb7c804199dbece4170370e2fc9135e5c6a1abe9b0f4b2fc"} Jul 05 11:53:22.744453 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.744412 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:22.744667 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.744479 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pwx6\" (UniqueName: \"kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6\") pod \"network-check-target-rt4k7\" (UID: \"3997821f-4556-407b-9b73-daf87e87f9b1\") " pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:22.744667 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:22.744536 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:22.744667 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:22.744657 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:22.744841 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:22.744723 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs podName:793d5b93-3b3a-4f7b-99fb-38db6730bfa6 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:24.744702132 +0000 UTC m=+5.135403881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs") pod "network-metrics-daemon-dhzh9" (UID: "793d5b93-3b3a-4f7b-99fb-38db6730bfa6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:22.745176 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:22.745153 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:22.745259 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:22.745211 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret podName:746ca75a-b87d-4153-9e6a-74845453758c nodeName:}" failed. No retries permitted until 2026-07-05 11:53:24.745194257 +0000 UTC m=+5.135896020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret") pod "global-pull-secret-syncer-8z5fg" (UID: "746ca75a-b87d-4153-9e6a-74845453758c") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:22.745469 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:22.745293 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:22.745469 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:22.745308 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:22.745469 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:22.745320 2575 projected.go:196] Error preparing data for projected volume kube-api-access-2pwx6 for pod openshift-network-diagnostics/network-check-target-rt4k7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:22.745469 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:22.745358 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6 podName:3997821f-4556-407b-9b73-daf87e87f9b1 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:24.745346209 +0000 UTC m=+5.136047966 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pwx6" (UniqueName: "kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6") pod "network-check-target-rt4k7" (UID: "3997821f-4556-407b-9b73-daf87e87f9b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:23.075011 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:23.074935 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 11:48:21 +0000 UTC" deadline="2027-12-06 06:45:34.069652459 +0000 UTC" Jul 05 11:53:23.075011 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:23.075006 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12450h52m10.994650866s" Jul 05 11:53:23.123502 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:23.123466 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:23.123695 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:23.123607 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:23.124055 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:23.124035 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:23.124170 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:23.124132 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:24.044423 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:24.044387 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:24.128760 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:24.128258 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:24.128760 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:24.128396 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:24.762337 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:24.762015 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:24.762337 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:24.762070 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:24.762337 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:24.762111 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pwx6\" (UniqueName: \"kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6\") pod \"network-check-target-rt4k7\" (UID: \"3997821f-4556-407b-9b73-daf87e87f9b1\") " pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:24.762337 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:24.762196 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:24.762337 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:24.762252 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:24.762337 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:24.762270 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs podName:793d5b93-3b3a-4f7b-99fb-38db6730bfa6 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:28.762249116 +0000 UTC m=+9.152950860 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs") pod "network-metrics-daemon-dhzh9" (UID: "793d5b93-3b3a-4f7b-99fb-38db6730bfa6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:24.762337 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:24.762316 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret podName:746ca75a-b87d-4153-9e6a-74845453758c nodeName:}" failed. No retries permitted until 2026-07-05 11:53:28.762295348 +0000 UTC m=+9.152997107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret") pod "global-pull-secret-syncer-8z5fg" (UID: "746ca75a-b87d-4153-9e6a-74845453758c") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:24.762337 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:24.762254 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:24.762337 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:24.762346 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:24.762900 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:24.762359 2575 projected.go:196] Error preparing data for projected volume kube-api-access-2pwx6 for pod openshift-network-diagnostics/network-check-target-rt4k7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:24.762900 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:24.762399 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6 podName:3997821f-4556-407b-9b73-daf87e87f9b1 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:28.762385298 +0000 UTC m=+9.153087056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pwx6" (UniqueName: "kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6") pod "network-check-target-rt4k7" (UID: "3997821f-4556-407b-9b73-daf87e87f9b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:25.123351 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:25.123144 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:25.123527 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:25.123429 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:25.123592 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:25.123175 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:25.123694 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:25.123668 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:26.130716 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:26.130637 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:26.131425 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:26.130779 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:27.123022 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:27.122940 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:27.123022 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:27.122989 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:27.123302 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:27.123088 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:27.123563 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:27.123521 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:28.127300 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:28.127268 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:28.127801 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:28.127390 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:28.797569 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:28.797483 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:28.797569 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:28.797528 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:28.797569 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:28.797567 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pwx6\" (UniqueName: \"kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6\") pod \"network-check-target-rt4k7\" (UID: \"3997821f-4556-407b-9b73-daf87e87f9b1\") " pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:28.797853 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:28.797676 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:28.797853 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:28.797706 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:28.797853 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:28.797725 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:28.797853 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:28.797736 2575 projected.go:196] Error preparing data for projected volume kube-api-access-2pwx6 for pod openshift-network-diagnostics/network-check-target-rt4k7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:28.797853 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:28.797676 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:28.797853 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:28.797770 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret podName:746ca75a-b87d-4153-9e6a-74845453758c nodeName:}" failed. No retries permitted until 2026-07-05 11:53:36.797748705 +0000 UTC m=+17.188450451 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret") pod "global-pull-secret-syncer-8z5fg" (UID: "746ca75a-b87d-4153-9e6a-74845453758c") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:28.797853 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:28.797796 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs podName:793d5b93-3b3a-4f7b-99fb-38db6730bfa6 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:36.797779828 +0000 UTC m=+17.188481593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs") pod "network-metrics-daemon-dhzh9" (UID: "793d5b93-3b3a-4f7b-99fb-38db6730bfa6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:28.797853 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:28.797823 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6 podName:3997821f-4556-407b-9b73-daf87e87f9b1 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:36.797813045 +0000 UTC m=+17.188514802 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pwx6" (UniqueName: "kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6") pod "network-check-target-rt4k7" (UID: "3997821f-4556-407b-9b73-daf87e87f9b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:29.123671 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:29.123638 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:29.123671 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:29.123660 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:29.123970 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:29.123772 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:29.123970 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:29.123862 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:30.124973 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:30.124940 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:30.125411 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:30.125085 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:31.123361 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:31.123323 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:31.123563 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:31.123364 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:31.123563 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:31.123459 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:31.123685 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:31.123581 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:32.123227 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:32.123191 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:32.123693 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:32.123340 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:33.123653 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:33.123619 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:33.124123 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:33.123619 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:33.124123 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:33.123755 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:33.124123 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:33.123796 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:34.122940 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:34.122905 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:34.123134 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:34.123066 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:35.122998 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:35.122953 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:35.122998 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:35.122981 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:35.123495 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:35.123076 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:35.123495 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:35.123272 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:36.123398 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:36.123361 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:36.123862 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:36.123501 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:36.855373 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:36.855334 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:36.855554 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:36.855385 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:36.855554 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:36.855423 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pwx6\" (UniqueName: \"kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6\") pod \"network-check-target-rt4k7\" (UID: \"3997821f-4556-407b-9b73-daf87e87f9b1\") " pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:36.855554 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:36.855473 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:36.855554 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:36.855532 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs podName:793d5b93-3b3a-4f7b-99fb-38db6730bfa6 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:52.855513566 +0000 UTC m=+33.246215309 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs") pod "network-metrics-daemon-dhzh9" (UID: "793d5b93-3b3a-4f7b-99fb-38db6730bfa6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:36.855554 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:36.855537 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:36.855554 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:36.855551 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:36.855554 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:36.855552 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:36.855891 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:36.855564 2575 projected.go:196] Error preparing data for projected volume kube-api-access-2pwx6 for pod openshift-network-diagnostics/network-check-target-rt4k7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:36.855891 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:36.855602 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret podName:746ca75a-b87d-4153-9e6a-74845453758c nodeName:}" failed. No retries permitted until 2026-07-05 11:53:52.855585792 +0000 UTC m=+33.246287547 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret") pod "global-pull-secret-syncer-8z5fg" (UID: "746ca75a-b87d-4153-9e6a-74845453758c") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:36.855891 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:36.855624 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6 podName:3997821f-4556-407b-9b73-daf87e87f9b1 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:52.85561435 +0000 UTC m=+33.246316104 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pwx6" (UniqueName: "kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6") pod "network-check-target-rt4k7" (UID: "3997821f-4556-407b-9b73-daf87e87f9b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:37.123650 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:37.123625 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:37.124045 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:37.123636 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:37.124045 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:37.123724 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:37.124045 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:37.123786 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:37.352732 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:37.352697 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9mt88" event={"ID":"161771f1-5ef9-49fd-924f-a9a89873ebce","Type":"ContainerStarted","Data":"03cf76d1d642a4e7acd2a5ea908c2a0f5eba5369a3f2719a5ed8a8fdb5b99db5"} Jul 05 11:53:37.354290 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:37.354143 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-207.ec2.internal" event={"ID":"3494324bd81becfa329adf53f8dc35af","Type":"ContainerStarted","Data":"47953b0a64676d93ab17de1cfa20046d018e420d34487bd6d776823106748d50"} Jul 05 11:53:37.371922 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:37.371822 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9mt88" podStartSLOduration=1.808296068 podStartE2EDuration="17.37180795s" totalImagesPullingTime="15.563511882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:37.371420466 +0000 UTC m=+17.762122228" watchObservedRunningTime="2026-07-05 11:53:37.37180795 +0000 UTC m=+17.762509711" Jul 05 11:53:37.386065 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:37.385998 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-207.ec2.internal" podStartSLOduration=16.385971017 podStartE2EDuration="16.385971017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:37.384854072 +0000 UTC m=+17.775555834" watchObservedRunningTime="2026-07-05 11:53:37.385971017 +0000 UTC m=+17.776672780" Jul 05 11:53:38.123806 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.123381 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:38.124510 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:38.123941 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:38.358352 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.358310 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s2dpw" event={"ID":"afa4b888-1bc8-4865-911e-5720c5b0028a","Type":"ContainerStarted","Data":"d34fef31efed3d1e41f27dff32fb6105ec10d005b89446b194fca8a26b89e34b"} Jul 05 11:53:38.359906 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.359859 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j6nrt" event={"ID":"d473ee62-6227-440a-bad0-ef141bb75628","Type":"ContainerStarted","Data":"8a8c51659cd609474a56baf9e0ebac023528e1b6c3302d3170a6301d63db4d6e"} Jul 05 11:53:38.361323 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.361302 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hb68f" event={"ID":"9930e6e9-d5bc-4384-a9de-97740b2b692d","Type":"ContainerStarted","Data":"28ff6264b094253d33b5f7691c73cb75472cf0ef3a5ae884a6303f49cba92c64"} Jul 05 11:53:38.364919 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.364894 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" event={"ID":"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf","Type":"ContainerStarted","Data":"77f2f2ded574354db26a720ea34c0abdcce47cabdf3664af1c9375b3d36112eb"} Jul 05 11:53:38.365036 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.364971 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" event={"ID":"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf","Type":"ContainerStarted","Data":"f22a046e39593a33b5c384a5e521ac338deba4055a2a240228019130517d5566"} Jul 05 11:53:38.365036 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.365017 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" event={"ID":"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf","Type":"ContainerStarted","Data":"8a775513b4a532851a0c6cb205b51dd8ad34a24f226a0e5594c7a075c2f28a96"} Jul 05 11:53:38.365151 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.365061 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" event={"ID":"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf","Type":"ContainerStarted","Data":"11427728e26c3bc8b67476804cf4356c43de01d47cba622e2292a347f0800ce6"} Jul 05 11:53:38.365151 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.365107 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" event={"ID":"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf","Type":"ContainerStarted","Data":"4aacbd186fe5ca6443ac7fb1a546df78545249c7c059965838ee7d3898f63da0"} Jul 05 11:53:38.365151 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.365149 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" event={"ID":"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf","Type":"ContainerStarted","Data":"d9aed719dc962d97117db6ca2e650d8848cf3c38d8198f0013537cd53d11f458"} Jul 05 11:53:38.366590 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.366567 2575 generic.go:356] "Generic (PLEG): container finished" podID="43726046-5603-4270-94d8-4e3f27f984b7" containerID="7765268b4932a4493c9d214d17f8b819cef1f0f34a66b0176cb7d1999a32780d" exitCode=0 Jul 05 11:53:38.366693 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.366655 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bc4j" event={"ID":"43726046-5603-4270-94d8-4e3f27f984b7","Type":"ContainerDied","Data":"7765268b4932a4493c9d214d17f8b819cef1f0f34a66b0176cb7d1999a32780d"} Jul 05 11:53:38.368513 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.368491 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rmzcs" event={"ID":"4ff7494e-12c6-42da-af77-38131222c9bf","Type":"ContainerStarted","Data":"9c75285306b1a6105a6d78d398cf8c060ee2b578fa5fbc26ca23fcd6dd3037be"} Jul 05 11:53:38.370208 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.370185 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" event={"ID":"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa","Type":"ContainerStarted","Data":"60550f29d008afc2ce4ec16572cae832325c31e672b382b41a93b579f2e8d64e"} Jul 05 11:53:38.371798 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.371748 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-s2dpw" podStartSLOduration=1.877308813 podStartE2EDuration="17.371731046s" totalImagesPullingTime="15.494422233s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:38.371466045 +0000 UTC m=+18.762167808" watchObservedRunningTime="2026-07-05 11:53:38.371731046 +0000 UTC m=+18.762432811" Jul 05 11:53:38.371922 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.371800 2575 generic.go:356] "Generic (PLEG): container finished" podID="03c8ac1e9b1e3df4f5ab2b1adae689d2" containerID="d76e63cf8614ea4b5cec1fca921a26ac534f6836c2cdbb5221697e3faf8949ce" exitCode=0 Jul 05 11:53:38.371922 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.371893 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" event={"ID":"03c8ac1e9b1e3df4f5ab2b1adae689d2","Type":"ContainerDied","Data":"d76e63cf8614ea4b5cec1fca921a26ac534f6836c2cdbb5221697e3faf8949ce"} Jul 05 11:53:38.385095 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.385058 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hb68f" podStartSLOduration=2.836494869 podStartE2EDuration="18.385044622s" totalImagesPullingTime="15.548549753s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:38.384716437 +0000 UTC m=+18.775418201" watchObservedRunningTime="2026-07-05 11:53:38.385044622 +0000 UTC m=+18.775746385" Jul 05 11:53:38.403210 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.403135 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rmzcs" podStartSLOduration=1.901020877 podStartE2EDuration="17.403115495s" totalImagesPullingTime="15.502094618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:38.402723326 +0000 UTC m=+18.793425089" watchObservedRunningTime="2026-07-05 11:53:38.403115495 +0000 UTC m=+18.793817259" Jul 05 11:53:38.437215 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.437158 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j6nrt" podStartSLOduration=1.909862398 podStartE2EDuration="17.437142915s" totalImagesPullingTime="15.527280517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:38.436933783 +0000 UTC m=+18.827635546" watchObservedRunningTime="2026-07-05 11:53:38.437142915 +0000 UTC m=+18.827844676" Jul 05 11:53:38.781822 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:38.781798 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 05 11:53:39.096818 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:39.096714 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-05T11:53:38.78181891Z","UUID":"471ca462-37a1-42b2-9679-11f9e624e974","Handler":null,"Name":"","Endpoint":""} Jul 05 11:53:39.098424 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:39.098400 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 05 11:53:39.098568 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:39.098444 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 05 11:53:39.123427 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:39.123398 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:39.123620 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:39.123516 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:39.123620 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:39.123565 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:39.123735 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:39.123643 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:39.377087 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:39.377001 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" event={"ID":"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa","Type":"ContainerStarted","Data":"d707e60fd3ef6f49187f470448835618541b9fbc15b2955d4d73014c5f608271"} Jul 05 11:53:39.379738 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:39.379341 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" event={"ID":"03c8ac1e9b1e3df4f5ab2b1adae689d2","Type":"ContainerStarted","Data":"9886c4bf6149c65f1e4321a13071c8b310a987f4fc4f9e54b1856f6e84ce6532"} Jul 05 11:53:39.393707 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:39.393186 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-207.ec2.internal" podStartSLOduration=18.39316395 podStartE2EDuration="18.39316395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:39.392794371 +0000 UTC m=+19.783496135" watchObservedRunningTime="2026-07-05 11:53:39.39316395 +0000 UTC m=+19.783865724" Jul 05 11:53:40.124400 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:40.124287 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:40.124400 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:40.124391 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:40.383831 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:40.383743 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" event={"ID":"82fdc3d7-4fd4-40ef-8c51-4b18c36aa2aa","Type":"ContainerStarted","Data":"b66c66f2290dcc56718bf5622b3c4081f6863a78011d9f38d268818b2b0e28af"} Jul 05 11:53:40.387351 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:40.387322 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" event={"ID":"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf","Type":"ContainerStarted","Data":"98304436d20e8889c8ac1c94861bc8f36f279671b9acf991318cb3dcf70367ab"} Jul 05 11:53:40.399415 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:40.399370 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l7xgj" podStartSLOduration=1.832882831 podStartE2EDuration="19.399356533s" totalImagesPullingTime="17.566473702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:40.399086352 +0000 UTC m=+20.789788113" watchObservedRunningTime="2026-07-05 11:53:40.399356533 +0000 UTC m=+20.790058294" Jul 05 11:53:41.123629 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:41.123594 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:41.123818 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:41.123603 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:41.123818 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:41.123701 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:41.123818 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:41.123781 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:42.003536 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:42.003501 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:42.004303 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:42.004277 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:42.124377 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:42.123837 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:42.124377 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:42.123991 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:42.398401 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:42.398225 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" event={"ID":"447db484-5c4b-4e4e-86f7-acbcb5c4f9cf","Type":"ContainerStarted","Data":"9ae5ff0897a143cb3a896fdcba2e0aa27ae7794d2f08c271c1bf7af47b9b4956"} Jul 05 11:53:42.398589 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:42.398460 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:42.398977 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:42.398959 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j6nrt" Jul 05 11:53:42.426791 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:42.426195 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" podStartSLOduration=6.341406263 podStartE2EDuration="22.426176583s" totalImagesPullingTime="16.08477032s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:42.42489624 +0000 UTC m=+22.815597996" watchObservedRunningTime="2026-07-05 11:53:42.426176583 +0000 UTC m=+22.816878346" Jul 05 11:53:43.123719 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:43.123636 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:43.123719 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:43.123674 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:43.124619 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:43.123768 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:43.124619 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:43.124195 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:43.400484 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:43.400396 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 05 11:53:43.400746 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:43.400632 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:43.401137 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:43.401113 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:43.421644 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:43.421364 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:43.425371 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:43.425017 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:43.500079 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:43.500036 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:53:44.119323 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:44.119294 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8z5fg"] Jul 05 11:53:44.119462 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:44.119411 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:44.119552 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:44.119523 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:44.122087 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:44.122062 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dhzh9"] Jul 05 11:53:44.122218 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:44.122191 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:44.122337 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:44.122312 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:44.122948 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:44.122866 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rt4k7"] Jul 05 11:53:44.123114 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:44.123076 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:44.123300 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:44.123236 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:44.405151 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:44.404918 2575 generic.go:356] "Generic (PLEG): container finished" podID="43726046-5603-4270-94d8-4e3f27f984b7" containerID="948b150c1189ee4806c7c8e78bf1ba3429946afc7f98e00bd8a2851e25046b53" exitCode=0 Jul 05 11:53:44.405585 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:44.405004 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bc4j" event={"ID":"43726046-5603-4270-94d8-4e3f27f984b7","Type":"ContainerDied","Data":"948b150c1189ee4806c7c8e78bf1ba3429946afc7f98e00bd8a2851e25046b53"} Jul 05 11:53:46.123064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:46.123034 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:46.123548 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:46.123034 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:46.123548 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:46.123163 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:46.123548 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:46.123034 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:46.123548 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:46.123254 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:46.123548 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:46.123342 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:48.126483 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:48.126449 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:48.127042 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:48.126567 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:48.127042 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:48.126652 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:48.127042 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:48.126789 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:48.127042 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:48.126836 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:48.127042 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:48.126928 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:49.418509 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:49.418346 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gjqwd" event={"ID":"9163d99b-81f1-49d9-85c4-e35200fb1b1c","Type":"ContainerStarted","Data":"b4c85f94b9dbde0011f65b72d8a2e804cd5ba7a0cfa289748c298a16fdc1f55e"} Jul 05 11:53:49.445362 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:49.445307 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gjqwd" podStartSLOduration=1.301804035 podStartE2EDuration="28.445286613s" totalImagesPullingTime="27.143482578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-05 11:53:49.444707711 +0000 UTC m=+29.835409474" watchObservedRunningTime="2026-07-05 11:53:49.445286613 +0000 UTC m=+29.835988375" Jul 05 11:53:50.126220 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.126189 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:50.126411 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.126195 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:50.126411 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:50.126274 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rt4k7" podUID="3997821f-4556-407b-9b73-daf87e87f9b1" Jul 05 11:53:50.126411 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.126288 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:50.126411 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:50.126378 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dhzh9" podUID="793d5b93-3b3a-4f7b-99fb-38db6730bfa6" Jul 05 11:53:50.126633 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:50.126444 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8z5fg" podUID="746ca75a-b87d-4153-9e6a-74845453758c" Jul 05 11:53:50.370604 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.370576 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-207.ec2.internal" event="NodeReady" Jul 05 11:53:50.370864 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.370694 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 05 11:53:50.399896 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.399846 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kh5kf"] Jul 05 11:53:50.423065 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.423034 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-96db66bd7-drwx2"] Jul 05 11:53:50.423727 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.423127 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kh5kf" Jul 05 11:53:50.423789 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.423400 2575 generic.go:356] "Generic (PLEG): container finished" podID="43726046-5603-4270-94d8-4e3f27f984b7" containerID="45017b697a47c1c074ff384416f3f55e3dac46a5fed1e70562095d9323539302" exitCode=0 Jul 05 11:53:50.427749 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.427725 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 05 11:53:50.428241 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.427829 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 05 11:53:50.428241 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.428045 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bh8tv\"" Jul 05 11:53:50.439915 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.439861 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bc4j" event={"ID":"43726046-5603-4270-94d8-4e3f27f984b7","Type":"ContainerDied","Data":"45017b697a47c1c074ff384416f3f55e3dac46a5fed1e70562095d9323539302"} Jul 05 11:53:50.440068 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.439934 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx"] Jul 05 11:53:50.440068 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.439980 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.442631 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.442606 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 05 11:53:50.442631 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.442623 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mbvkr\"" Jul 05 11:53:50.442983 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.442961 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 05 11:53:50.443076 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.442965 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 05 11:53:50.448421 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.448399 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 05 11:53:50.459549 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.459520 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr"] Jul 05 11:53:50.459660 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.459643 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" Jul 05 11:53:50.462484 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.462458 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 05 11:53:50.462623 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.462458 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 05 11:53:50.462623 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.462464 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 05 11:53:50.462623 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.462582 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-fb2lf\"" Jul 05 11:53:50.462808 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.462689 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 05 11:53:50.483477 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.483449 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q"] Jul 05 11:53:50.483663 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.483584 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:53:50.486358 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.486339 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 05 11:53:50.486697 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.486679 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 05 11:53:50.487001 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.486985 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 05 11:53:50.487116 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.487100 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zmtrj\"" Jul 05 11:53:50.498271 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.498248 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-4sth7"] Jul 05 11:53:50.498378 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.498358 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" Jul 05 11:53:50.500999 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.500979 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 05 11:53:50.501276 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.501259 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 05 11:53:50.501337 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.501315 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 05 11:53:50.501399 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.501337 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-nknhj\"" Jul 05 11:53:50.501466 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.501451 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 05 11:53:50.522490 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.522462 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-kf6hq"] Jul 05 11:53:50.522648 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.522588 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.525485 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.525462 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 05 11:53:50.525624 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.525485 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 05 11:53:50.525624 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.525464 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2q4bx\"" Jul 05 11:53:50.525624 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.525519 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 05 11:53:50.525795 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.525783 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 05 11:53:50.531009 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.530989 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 05 11:53:50.546764 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.546729 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-768g2"] Jul 05 11:53:50.546937 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.546844 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.550060 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.550019 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 05 11:53:50.550228 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.550055 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fxnj7\"" Jul 05 11:53:50.550228 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.550181 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 05 11:53:50.551035 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.551015 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 05 11:53:50.551137 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.551020 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 05 11:53:50.554324 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.554292 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:53:50.556134 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556105 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmxph\" (UniqueName: \"kubernetes.io/projected/be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34-kube-api-access-dmxph\") pod \"kube-storage-version-migrator-operator-846bf668f7-j8f2q\" (UID: \"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" Jul 05 11:53:50.556240 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556189 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dde2fd-fba0-412f-90a3-97cc45e9c537-ca-trust-extracted\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.556240 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-certificates\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.556339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556277 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-j8f2q\" (UID: \"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" Jul 05 11:53:50.556339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556301 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-installation-pull-secrets\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.556442 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556337 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1881b431-0b27-4e0f-a16f-6a0e93cafaa3-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-md6wx\" (UID: \"1881b431-0b27-4e0f-a16f-6a0e93cafaa3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" Jul 05 11:53:50.556442 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556380 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-bound-sa-token\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.556534 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556471 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:53:50.556615 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556590 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-j8f2q\" (UID: \"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" Jul 05 11:53:50.556669 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556624 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-image-registry-private-configuration\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.556669 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556649 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-trusted-ca\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.556765 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556672 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q2tp\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-kube-api-access-2q2tp\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.556765 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556716 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67sth\" (UniqueName: \"kubernetes.io/projected/f3da3ff4-6df1-4407-8fd1-02ef70c431b6-kube-api-access-67sth\") pod \"volume-data-source-validator-5d456fdbcd-kh5kf\" (UID: \"f3da3ff4-6df1-4407-8fd1-02ef70c431b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kh5kf" Jul 05 11:53:50.556765 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556731 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wr7d\" (UniqueName: \"kubernetes.io/projected/1881b431-0b27-4e0f-a16f-6a0e93cafaa3-kube-api-access-8wr7d\") pod \"service-ca-operator-5d46d9ffd4-md6wx\" (UID: \"1881b431-0b27-4e0f-a16f-6a0e93cafaa3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" Jul 05 11:53:50.556939 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556776 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.556939 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556809 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54dhn\" (UniqueName: \"kubernetes.io/projected/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-kube-api-access-54dhn\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:53:50.556939 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.556840 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1881b431-0b27-4e0f-a16f-6a0e93cafaa3-config\") pod \"service-ca-operator-5d46d9ffd4-md6wx\" (UID: \"1881b431-0b27-4e0f-a16f-6a0e93cafaa3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" Jul 05 11:53:50.557176 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.557158 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 05 11:53:50.562193 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.562177 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kh5kf"] Jul 05 11:53:50.562273 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.562205 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gplkw"] Jul 05 11:53:50.562305 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.562278 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-768g2" Jul 05 11:53:50.564652 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.564634 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-9nzkk\"" Jul 05 11:53:50.564827 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.564814 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 11:53:50.564939 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.564924 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 11:53:50.580397 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.580364 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-f67t2"] Jul 05 11:53:50.580546 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.580444 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.583574 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.583553 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 05 11:53:50.583704 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.583605 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gbwbs\"" Jul 05 11:53:50.583704 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.583624 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 05 11:53:50.600445 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.600426 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-768g2"] Jul 05 11:53:50.600554 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.600455 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx"] Jul 05 11:53:50.600554 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.600469 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q"] Jul 05 11:53:50.600554 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.600483 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-4sth7"] Jul 05 11:53:50.600554 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.600501 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-96db66bd7-drwx2"] Jul 05 11:53:50.600554 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.600514 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f67t2"] Jul 05 11:53:50.600554 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.600526 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gplkw"] Jul 05 11:53:50.600554 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.600540 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr"] Jul 05 11:53:50.600554 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.600555 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-kf6hq"] Jul 05 11:53:50.600826 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.600626 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f67t2" Jul 05 11:53:50.603378 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.603360 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 05 11:53:50.603466 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.603360 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c4hs5\"" Jul 05 11:53:50.603466 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.603360 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 05 11:53:50.603546 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.603483 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 05 11:53:50.633327 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.633245 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:53:50.658170 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658144 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7197549e-9314-42dd-bbdb-d53feb6ece0e-tmp\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.658287 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658181 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmxph\" (UniqueName: \"kubernetes.io/projected/be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34-kube-api-access-dmxph\") pod \"kube-storage-version-migrator-operator-846bf668f7-j8f2q\" (UID: \"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" Jul 05 11:53:50.658287 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658203 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-certificates\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.658287 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658253 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcjt6\" (UniqueName: \"kubernetes.io/projected/4b444ec4-9b23-422d-ac9d-dd3535492d17-kube-api-access-dcjt6\") pod \"ingress-canary-f67t2\" (UID: \"4b444ec4-9b23-422d-ac9d-dd3535492d17\") " pod="openshift-ingress-canary/ingress-canary-f67t2" Jul 05 11:53:50.658388 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658285 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31952da1-f50a-4817-b342-775273f214f5-config-volume\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.658388 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658331 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-j8f2q\" (UID: \"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" Jul 05 11:53:50.658388 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658358 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7197549e-9314-42dd-bbdb-d53feb6ece0e-snapshots\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.658388 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658379 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.658546 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658399 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7197549e-9314-42dd-bbdb-d53feb6ece0e-serving-cert\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.658546 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658418 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1881b431-0b27-4e0f-a16f-6a0e93cafaa3-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-md6wx\" (UID: \"1881b431-0b27-4e0f-a16f-6a0e93cafaa3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" Jul 05 11:53:50.658546 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658435 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-bound-sa-token\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.658546 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658453 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7197549e-9314-42dd-bbdb-d53feb6ece0e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.658546 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658477 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b444ec4-9b23-422d-ac9d-dd3535492d17-cert\") pod \"ingress-canary-f67t2\" (UID: \"4b444ec4-9b23-422d-ac9d-dd3535492d17\") " pod="openshift-ingress-canary/ingress-canary-f67t2" Jul 05 11:53:50.658546 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658529 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f7f19ec-9537-4581-b1ad-62d17debda03-serving-cert\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.658839 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658581 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-j8f2q\" (UID: \"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" Jul 05 11:53:50.658839 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658614 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q2tp\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-kube-api-access-2q2tp\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.658839 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658644 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzct8\" (UniqueName: \"kubernetes.io/projected/31952da1-f50a-4817-b342-775273f214f5-kube-api-access-bzct8\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.658839 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658691 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-image-registry-private-configuration\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.659048 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658841 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67sth\" (UniqueName: \"kubernetes.io/projected/f3da3ff4-6df1-4407-8fd1-02ef70c431b6-kube-api-access-67sth\") pod \"volume-data-source-validator-5d456fdbcd-kh5kf\" (UID: \"f3da3ff4-6df1-4407-8fd1-02ef70c431b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kh5kf" Jul 05 11:53:50.659048 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658902 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wr7d\" (UniqueName: \"kubernetes.io/projected/1881b431-0b27-4e0f-a16f-6a0e93cafaa3-kube-api-access-8wr7d\") pod \"service-ca-operator-5d46d9ffd4-md6wx\" (UID: \"1881b431-0b27-4e0f-a16f-6a0e93cafaa3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" Jul 05 11:53:50.659048 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658938 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.659048 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.658966 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-j8f2q\" (UID: \"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" Jul 05 11:53:50.659048 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659011 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31952da1-f50a-4817-b342-775273f214f5-tmp-dir\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.659278 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659054 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-certificates\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.659278 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659065 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dde2fd-fba0-412f-90a3-97cc45e9c537-ca-trust-extracted\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.659278 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659117 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-installation-pull-secrets\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.659278 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659148 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsvp5\" (UniqueName: \"kubernetes.io/projected/7197549e-9314-42dd-bbdb-d53feb6ece0e-kube-api-access-dsvp5\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.659278 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:50.659120 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 11:53:50.659278 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659178 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84c7n\" (UniqueName: \"kubernetes.io/projected/5f7f19ec-9537-4581-b1ad-62d17debda03-kube-api-access-84c7n\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.659278 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:50.659192 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96db66bd7-drwx2: secret "image-registry-tls" not found Jul 05 11:53:50.659278 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7197549e-9314-42dd-bbdb-d53feb6ece0e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.659278 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659231 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52x5c\" (UniqueName: \"kubernetes.io/projected/02ab04f0-b13c-4ee6-8646-715089510936-kube-api-access-52x5c\") pod \"network-check-source-989f8b8dc-768g2\" (UID: \"02ab04f0-b13c-4ee6-8646-715089510936\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-768g2" Jul 05 11:53:50.659278 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:50.659258 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls podName:05dde2fd-fba0-412f-90a3-97cc45e9c537 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:51.159239422 +0000 UTC m=+31.549941168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls") pod "image-registry-96db66bd7-drwx2" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537") : secret "image-registry-tls" not found Jul 05 11:53:50.659682 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659316 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:53:50.659682 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659348 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-trusted-ca\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.659682 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659377 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f7f19ec-9537-4581-b1ad-62d17debda03-trusted-ca\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.659682 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659407 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f7f19ec-9537-4581-b1ad-62d17debda03-config\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.659682 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659434 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dde2fd-fba0-412f-90a3-97cc45e9c537-ca-trust-extracted\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.659682 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659441 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54dhn\" (UniqueName: \"kubernetes.io/projected/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-kube-api-access-54dhn\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:53:50.659682 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.659468 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1881b431-0b27-4e0f-a16f-6a0e93cafaa3-config\") pod \"service-ca-operator-5d46d9ffd4-md6wx\" (UID: \"1881b431-0b27-4e0f-a16f-6a0e93cafaa3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" Jul 05 11:53:50.659682 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:50.659498 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 11:53:50.659682 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:50.659564 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls podName:8efd845e-7e3b-4a9f-9572-43d72a6a22f0 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:51.159548619 +0000 UTC m=+31.550250359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-479rr" (UID: "8efd845e-7e3b-4a9f-9572-43d72a6a22f0") : secret "samples-operator-tls" not found Jul 05 11:53:50.660064 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.660003 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1881b431-0b27-4e0f-a16f-6a0e93cafaa3-config\") pod \"service-ca-operator-5d46d9ffd4-md6wx\" (UID: \"1881b431-0b27-4e0f-a16f-6a0e93cafaa3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" Jul 05 11:53:50.660351 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.660330 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-trusted-ca\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.662931 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.662911 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-j8f2q\" (UID: \"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" Jul 05 11:53:50.663026 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.662943 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1881b431-0b27-4e0f-a16f-6a0e93cafaa3-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-md6wx\" (UID: \"1881b431-0b27-4e0f-a16f-6a0e93cafaa3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" Jul 05 11:53:50.663202 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.663181 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-image-registry-private-configuration\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.663281 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.663212 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-installation-pull-secrets\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.667709 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.667680 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmxph\" (UniqueName: \"kubernetes.io/projected/be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34-kube-api-access-dmxph\") pod \"kube-storage-version-migrator-operator-846bf668f7-j8f2q\" (UID: \"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" Jul 05 11:53:50.667801 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.667708 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wr7d\" (UniqueName: \"kubernetes.io/projected/1881b431-0b27-4e0f-a16f-6a0e93cafaa3-kube-api-access-8wr7d\") pod \"service-ca-operator-5d46d9ffd4-md6wx\" (UID: \"1881b431-0b27-4e0f-a16f-6a0e93cafaa3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" Jul 05 11:53:50.667905 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.667885 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67sth\" (UniqueName: \"kubernetes.io/projected/f3da3ff4-6df1-4407-8fd1-02ef70c431b6-kube-api-access-67sth\") pod \"volume-data-source-validator-5d456fdbcd-kh5kf\" (UID: \"f3da3ff4-6df1-4407-8fd1-02ef70c431b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kh5kf" Jul 05 11:53:50.668247 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.668227 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-bound-sa-token\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.668630 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.668609 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54dhn\" (UniqueName: \"kubernetes.io/projected/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-kube-api-access-54dhn\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:53:50.668668 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.668633 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q2tp\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-kube-api-access-2q2tp\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:50.741126 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.741089 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kh5kf" Jul 05 11:53:50.760581 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.760550 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7197549e-9314-42dd-bbdb-d53feb6ece0e-snapshots\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.760581 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.760583 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.760774 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.760601 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7197549e-9314-42dd-bbdb-d53feb6ece0e-serving-cert\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.760774 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:50.760689 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 11:53:50.760774 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.760710 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7197549e-9314-42dd-bbdb-d53feb6ece0e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.760774 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.760741 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b444ec4-9b23-422d-ac9d-dd3535492d17-cert\") pod \"ingress-canary-f67t2\" (UID: \"4b444ec4-9b23-422d-ac9d-dd3535492d17\") " pod="openshift-ingress-canary/ingress-canary-f67t2" Jul 05 11:53:50.760774 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:50.760761 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls podName:31952da1-f50a-4817-b342-775273f214f5 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:51.26073978 +0000 UTC m=+31.651441545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls") pod "dns-default-gplkw" (UID: "31952da1-f50a-4817-b342-775273f214f5") : secret "dns-default-metrics-tls" not found Jul 05 11:53:50.761077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.760790 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f7f19ec-9537-4581-b1ad-62d17debda03-serving-cert\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.761077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.760831 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzct8\" (UniqueName: \"kubernetes.io/projected/31952da1-f50a-4817-b342-775273f214f5-kube-api-access-bzct8\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.761077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.760921 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31952da1-f50a-4817-b342-775273f214f5-tmp-dir\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.761077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.760972 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsvp5\" (UniqueName: \"kubernetes.io/projected/7197549e-9314-42dd-bbdb-d53feb6ece0e-kube-api-access-dsvp5\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.761077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761004 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84c7n\" (UniqueName: \"kubernetes.io/projected/5f7f19ec-9537-4581-b1ad-62d17debda03-kube-api-access-84c7n\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.761077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761030 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7197549e-9314-42dd-bbdb-d53feb6ece0e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.761077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761058 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52x5c\" (UniqueName: \"kubernetes.io/projected/02ab04f0-b13c-4ee6-8646-715089510936-kube-api-access-52x5c\") pod \"network-check-source-989f8b8dc-768g2\" (UID: \"02ab04f0-b13c-4ee6-8646-715089510936\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-768g2" Jul 05 11:53:50.761414 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761326 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7197549e-9314-42dd-bbdb-d53feb6ece0e-snapshots\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.761659 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761630 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31952da1-f50a-4817-b342-775273f214f5-tmp-dir\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.761752 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761690 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f7f19ec-9537-4581-b1ad-62d17debda03-trusted-ca\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.761752 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761726 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f7f19ec-9537-4581-b1ad-62d17debda03-config\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.761856 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761772 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7197549e-9314-42dd-bbdb-d53feb6ece0e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.761856 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761777 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7197549e-9314-42dd-bbdb-d53feb6ece0e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.761978 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761886 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7197549e-9314-42dd-bbdb-d53feb6ece0e-tmp\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.761978 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761927 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcjt6\" (UniqueName: \"kubernetes.io/projected/4b444ec4-9b23-422d-ac9d-dd3535492d17-kube-api-access-dcjt6\") pod \"ingress-canary-f67t2\" (UID: \"4b444ec4-9b23-422d-ac9d-dd3535492d17\") " pod="openshift-ingress-canary/ingress-canary-f67t2" Jul 05 11:53:50.761978 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.761952 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31952da1-f50a-4817-b342-775273f214f5-config-volume\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.762182 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.762158 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7197549e-9314-42dd-bbdb-d53feb6ece0e-tmp\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.762560 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.762538 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31952da1-f50a-4817-b342-775273f214f5-config-volume\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.762624 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.762611 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f7f19ec-9537-4581-b1ad-62d17debda03-config\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.762720 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.762698 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f7f19ec-9537-4581-b1ad-62d17debda03-trusted-ca\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.763304 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.763288 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7197549e-9314-42dd-bbdb-d53feb6ece0e-serving-cert\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.763410 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.763395 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f7f19ec-9537-4581-b1ad-62d17debda03-serving-cert\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.763459 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.763442 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b444ec4-9b23-422d-ac9d-dd3535492d17-cert\") pod \"ingress-canary-f67t2\" (UID: \"4b444ec4-9b23-422d-ac9d-dd3535492d17\") " pod="openshift-ingress-canary/ingress-canary-f67t2" Jul 05 11:53:50.768292 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.768269 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" Jul 05 11:53:50.770057 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.770033 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzct8\" (UniqueName: \"kubernetes.io/projected/31952da1-f50a-4817-b342-775273f214f5-kube-api-access-bzct8\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:50.770159 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.770125 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsvp5\" (UniqueName: \"kubernetes.io/projected/7197549e-9314-42dd-bbdb-d53feb6ece0e-kube-api-access-dsvp5\") pod \"insights-operator-5c847dcfc4-kf6hq\" (UID: \"7197549e-9314-42dd-bbdb-d53feb6ece0e\") " pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.770566 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.770535 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcjt6\" (UniqueName: \"kubernetes.io/projected/4b444ec4-9b23-422d-ac9d-dd3535492d17-kube-api-access-dcjt6\") pod \"ingress-canary-f67t2\" (UID: \"4b444ec4-9b23-422d-ac9d-dd3535492d17\") " pod="openshift-ingress-canary/ingress-canary-f67t2" Jul 05 11:53:50.770660 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.770625 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84c7n\" (UniqueName: \"kubernetes.io/projected/5f7f19ec-9537-4581-b1ad-62d17debda03-kube-api-access-84c7n\") pod \"console-operator-68b757fb67-4sth7\" (UID: \"5f7f19ec-9537-4581-b1ad-62d17debda03\") " pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.771077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.771059 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52x5c\" (UniqueName: \"kubernetes.io/projected/02ab04f0-b13c-4ee6-8646-715089510936-kube-api-access-52x5c\") pod \"network-check-source-989f8b8dc-768g2\" (UID: \"02ab04f0-b13c-4ee6-8646-715089510936\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-768g2" Jul 05 11:53:50.806711 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.806683 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" Jul 05 11:53:50.832365 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.832341 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:53:50.859905 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.859708 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" Jul 05 11:53:50.871253 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.870964 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-768g2" Jul 05 11:53:50.908328 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.908253 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f67t2" Jul 05 11:53:50.987460 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:50.987390 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kh5kf"] Jul 05 11:53:51.000449 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.000103 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx"] Jul 05 11:53:51.035616 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.034927 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q"] Jul 05 11:53:51.042260 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:51.042224 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe6fbe1d_9c8a_4e98_bc0a_f244bd1bad34.slice/crio-651277a94aa5657b2e1afb54b6acf396b050f0d984d0405ca174369d5e738207 WatchSource:0}: Error finding container 651277a94aa5657b2e1afb54b6acf396b050f0d984d0405ca174369d5e738207: Status 404 returned error can't find the container with id 651277a94aa5657b2e1afb54b6acf396b050f0d984d0405ca174369d5e738207 Jul 05 11:53:51.048668 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.048529 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-4sth7"] Jul 05 11:53:51.111186 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.111149 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-kf6hq"] Jul 05 11:53:51.114575 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:51.114543 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7197549e_9314_42dd_bbdb_d53feb6ece0e.slice/crio-1f094b2dad766c7cabbcd2d0cf966935c8414c0881af904b6863e967f7de344c WatchSource:0}: Error finding container 1f094b2dad766c7cabbcd2d0cf966935c8414c0881af904b6863e967f7de344c: Status 404 returned error can't find the container with id 1f094b2dad766c7cabbcd2d0cf966935c8414c0881af904b6863e967f7de344c Jul 05 11:53:51.127137 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.127109 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-768g2"] Jul 05 11:53:51.132220 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:51.132176 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02ab04f0_b13c_4ee6_8646_715089510936.slice/crio-904ab22736afc3e55f4f5395de568a3407cae79d72caf4ccd548be823f472927 WatchSource:0}: Error finding container 904ab22736afc3e55f4f5395de568a3407cae79d72caf4ccd548be823f472927: Status 404 returned error can't find the container with id 904ab22736afc3e55f4f5395de568a3407cae79d72caf4ccd548be823f472927 Jul 05 11:53:51.143143 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.143079 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f67t2"] Jul 05 11:53:51.145773 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:53:51.145748 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b444ec4_9b23_422d_ac9d_dd3535492d17.slice/crio-380c1c5744c9eaeb5538aec8251b406d9c9bbc3d85ebcd1d5fb1bf4cc111789c WatchSource:0}: Error finding container 380c1c5744c9eaeb5538aec8251b406d9c9bbc3d85ebcd1d5fb1bf4cc111789c: Status 404 returned error can't find the container with id 380c1c5744c9eaeb5538aec8251b406d9c9bbc3d85ebcd1d5fb1bf4cc111789c Jul 05 11:53:51.165803 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.165777 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:51.165942 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.165858 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:53:51.166000 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:51.165943 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 11:53:51.166000 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:51.165960 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96db66bd7-drwx2: secret "image-registry-tls" not found Jul 05 11:53:51.166000 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:51.165973 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 11:53:51.166105 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:51.166011 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls podName:05dde2fd-fba0-412f-90a3-97cc45e9c537 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:52.165994623 +0000 UTC m=+32.556696363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls") pod "image-registry-96db66bd7-drwx2" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537") : secret "image-registry-tls" not found Jul 05 11:53:51.166105 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:51.166025 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls podName:8efd845e-7e3b-4a9f-9572-43d72a6a22f0 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:52.166018828 +0000 UTC m=+32.556720568 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-479rr" (UID: "8efd845e-7e3b-4a9f-9572-43d72a6a22f0") : secret "samples-operator-tls" not found Jul 05 11:53:51.267266 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.267227 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:51.267432 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:51.267398 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 11:53:51.267495 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:51.267469 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls podName:31952da1-f50a-4817-b342-775273f214f5 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:52.26745241 +0000 UTC m=+32.658154157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls") pod "dns-default-gplkw" (UID: "31952da1-f50a-4817-b342-775273f214f5") : secret "dns-default-metrics-tls" not found Jul 05 11:53:51.427372 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.427286 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-768g2" event={"ID":"02ab04f0-b13c-4ee6-8646-715089510936","Type":"ContainerStarted","Data":"904ab22736afc3e55f4f5395de568a3407cae79d72caf4ccd548be823f472927"} Jul 05 11:53:51.428358 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.428333 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" event={"ID":"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34","Type":"ContainerStarted","Data":"651277a94aa5657b2e1afb54b6acf396b050f0d984d0405ca174369d5e738207"} Jul 05 11:53:51.429348 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.429323 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f67t2" event={"ID":"4b444ec4-9b23-422d-ac9d-dd3535492d17","Type":"ContainerStarted","Data":"380c1c5744c9eaeb5538aec8251b406d9c9bbc3d85ebcd1d5fb1bf4cc111789c"} Jul 05 11:53:51.430328 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.430309 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" event={"ID":"5f7f19ec-9537-4581-b1ad-62d17debda03","Type":"ContainerStarted","Data":"bd127ad385963ef2459987fbf9a8f8d4091a88ab51b39043984a92be02264361"} Jul 05 11:53:51.431275 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.431257 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" event={"ID":"1881b431-0b27-4e0f-a16f-6a0e93cafaa3","Type":"ContainerStarted","Data":"beaff6e80c3536e726d6476a807321ccd0a61e598d247ccb207d4b8df63f70c2"} Jul 05 11:53:51.432128 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.432109 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" event={"ID":"7197549e-9314-42dd-bbdb-d53feb6ece0e","Type":"ContainerStarted","Data":"1f094b2dad766c7cabbcd2d0cf966935c8414c0881af904b6863e967f7de344c"} Jul 05 11:53:51.433011 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:51.432994 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kh5kf" event={"ID":"f3da3ff4-6df1-4407-8fd1-02ef70c431b6","Type":"ContainerStarted","Data":"90c5e12429b9146e40ddb5060af431c31f46d1b33eb31d5142969571fc983bd7"} Jul 05 11:53:52.126905 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.122913 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:52.126905 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.123347 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:52.126905 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.123753 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:52.126905 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.126816 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 11:53:52.130472 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.129932 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 11:53:52.130472 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.130223 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dsspw\"" Jul 05 11:53:52.131446 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.131261 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-759jc\"" Jul 05 11:53:52.176507 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.176474 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:53:52.176644 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.176595 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:52.176796 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:52.176778 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 11:53:52.176850 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:52.176797 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96db66bd7-drwx2: secret "image-registry-tls" not found Jul 05 11:53:52.176850 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:52.176850 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls podName:05dde2fd-fba0-412f-90a3-97cc45e9c537 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:54.176832141 +0000 UTC m=+34.567533887 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls") pod "image-registry-96db66bd7-drwx2" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537") : secret "image-registry-tls" not found Jul 05 11:53:52.177270 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:52.177251 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 11:53:52.177347 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:52.177306 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls podName:8efd845e-7e3b-4a9f-9572-43d72a6a22f0 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:54.177291843 +0000 UTC m=+34.567993590 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-479rr" (UID: "8efd845e-7e3b-4a9f-9572-43d72a6a22f0") : secret "samples-operator-tls" not found Jul 05 11:53:52.277503 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.277464 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:52.278837 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:52.278806 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 11:53:52.278985 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:52.278888 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls podName:31952da1-f50a-4817-b342-775273f214f5 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:54.27885469 +0000 UTC m=+34.669556433 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls") pod "dns-default-gplkw" (UID: "31952da1-f50a-4817-b342-775273f214f5") : secret "dns-default-metrics-tls" not found Jul 05 11:53:52.445697 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.445609 2575 generic.go:356] "Generic (PLEG): container finished" podID="43726046-5603-4270-94d8-4e3f27f984b7" containerID="2ad28b78cd682d95b575783e71416e68e088d4670fdad2545a0a9f7c44ade600" exitCode=0 Jul 05 11:53:52.446179 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.445695 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bc4j" event={"ID":"43726046-5603-4270-94d8-4e3f27f984b7","Type":"ContainerDied","Data":"2ad28b78cd682d95b575783e71416e68e088d4670fdad2545a0a9f7c44ade600"} Jul 05 11:53:52.884525 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.884484 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:53:52.884729 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.884540 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:52.884729 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.884580 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pwx6\" (UniqueName: \"kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6\") pod \"network-check-target-rt4k7\" (UID: \"3997821f-4556-407b-9b73-daf87e87f9b1\") " pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:52.885845 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:52.885351 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 05 11:53:52.885845 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:52.885625 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs podName:793d5b93-3b3a-4f7b-99fb-38db6730bfa6 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:24.88560431 +0000 UTC m=+65.276306064 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs") pod "network-metrics-daemon-dhzh9" (UID: "793d5b93-3b3a-4f7b-99fb-38db6730bfa6") : secret "metrics-daemon-secret" not found Jul 05 11:53:52.895055 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.894996 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/746ca75a-b87d-4153-9e6a-74845453758c-original-pull-secret\") pod \"global-pull-secret-syncer-8z5fg\" (UID: \"746ca75a-b87d-4153-9e6a-74845453758c\") " pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:52.897682 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:52.897656 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pwx6\" (UniqueName: \"kubernetes.io/projected/3997821f-4556-407b-9b73-daf87e87f9b1-kube-api-access-2pwx6\") pod \"network-check-target-rt4k7\" (UID: \"3997821f-4556-407b-9b73-daf87e87f9b1\") " pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:53.043759 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:53.043729 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:53:53.053067 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:53.053037 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8z5fg" Jul 05 11:53:53.283490 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:53.283397 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8z5fg"] Jul 05 11:53:53.293467 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:53.293441 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rt4k7"] Jul 05 11:53:53.454053 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:53.453684 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8z5fg" event={"ID":"746ca75a-b87d-4153-9e6a-74845453758c","Type":"ContainerStarted","Data":"ea86c5fdd5ad270bb418fe3c14d7687b875fc7204ac9e4dc68f83adbac1e6d08"} Jul 05 11:53:53.460482 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:53.460383 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rt4k7" event={"ID":"3997821f-4556-407b-9b73-daf87e87f9b1","Type":"ContainerStarted","Data":"6f357f79dde7d91000d914d770eb1f94f16f8ea0d6adce344b6e502e4f752c93"} Jul 05 11:53:53.464747 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:53.464010 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kh5kf" event={"ID":"f3da3ff4-6df1-4407-8fd1-02ef70c431b6","Type":"ContainerStarted","Data":"d55a9eb6505d180564ea18dc195ed78ae34396b17d669dcc1fcd54b793cb98b9"} Jul 05 11:53:53.483390 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:53.482463 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kh5kf" podStartSLOduration=23.3740936 podStartE2EDuration="25.482440577s" totalImagesPullingTime="2.108346977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:53.481581421 +0000 UTC m=+33.872283186" watchObservedRunningTime="2026-07-05 11:53:53.482440577 +0000 UTC m=+33.873142340" Jul 05 11:53:53.552463 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:53.552337 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:53:53.606674 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:53.606626 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:53:54.198504 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:54.198076 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:54.198504 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:54.198152 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:53:54.198504 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:54.198252 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 11:53:54.198504 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:54.198255 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 11:53:54.198504 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:54.198272 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96db66bd7-drwx2: secret "image-registry-tls" not found Jul 05 11:53:54.198504 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:54.198311 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls podName:8efd845e-7e3b-4a9f-9572-43d72a6a22f0 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:58.198291916 +0000 UTC m=+38.588993672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-479rr" (UID: "8efd845e-7e3b-4a9f-9572-43d72a6a22f0") : secret "samples-operator-tls" not found Jul 05 11:53:54.198504 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:54.198328 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls podName:05dde2fd-fba0-412f-90a3-97cc45e9c537 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:58.198319808 +0000 UTC m=+38.589021550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls") pod "image-registry-96db66bd7-drwx2" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537") : secret "image-registry-tls" not found Jul 05 11:53:54.298769 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:54.298723 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:54.298945 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:54.298925 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 11:53:54.299003 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:54.298990 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls podName:31952da1-f50a-4817-b342-775273f214f5 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:58.298971118 +0000 UTC m=+38.689672861 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls") pod "dns-default-gplkw" (UID: "31952da1-f50a-4817-b342-775273f214f5") : secret "dns-default-metrics-tls" not found Jul 05 11:53:55.830728 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:55.830635 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:53:55.880181 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:55.880119 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:53:58.233148 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:58.233111 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:53:58.233691 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:58.233183 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:53:58.233691 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:58.233289 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 11:53:58.233691 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:58.233312 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96db66bd7-drwx2: secret "image-registry-tls" not found Jul 05 11:53:58.233691 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:58.233341 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 11:53:58.233691 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:58.233375 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls podName:05dde2fd-fba0-412f-90a3-97cc45e9c537 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:06.233355158 +0000 UTC m=+46.624056911 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls") pod "image-registry-96db66bd7-drwx2" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537") : secret "image-registry-tls" not found Jul 05 11:53:58.233691 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:58.233405 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls podName:8efd845e-7e3b-4a9f-9572-43d72a6a22f0 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:06.233385505 +0000 UTC m=+46.624087260 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-479rr" (UID: "8efd845e-7e3b-4a9f-9572-43d72a6a22f0") : secret "samples-operator-tls" not found Jul 05 11:53:58.334590 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:58.334556 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:53:58.334783 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:58.334740 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 11:53:58.334856 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:53:58.334817 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls podName:31952da1-f50a-4817-b342-775273f214f5 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:06.334796686 +0000 UTC m=+46.725498427 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls") pod "dns-default-gplkw" (UID: "31952da1-f50a-4817-b342-775273f214f5") : secret "dns-default-metrics-tls" not found Jul 05 11:53:59.623722 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:59.623682 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:53:59.672257 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:53:59.672232 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:00.637467 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:00.637406 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:00.693778 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:00.693729 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:02.490529 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:02.490485 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" event={"ID":"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34","Type":"ContainerStarted","Data":"3c721e69a435f947f3adb911eabdf396c1b5b8770cc7599dbb7dd95795eebd84"} Jul 05 11:54:02.505627 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:02.505567 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" podStartSLOduration=24.061758625 podStartE2EDuration="34.50555001s" totalImagesPullingTime="10.443791385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:02.50519716 +0000 UTC m=+42.895898932" watchObservedRunningTime="2026-07-05 11:54:02.50555001 +0000 UTC m=+42.896251750" Jul 05 11:54:03.293501 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.293462 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:03.305628 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.305575 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh"] Jul 05 11:54:03.333395 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.333357 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh"] Jul 05 11:54:03.333557 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.333501 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh" Jul 05 11:54:03.336296 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.336268 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 05 11:54:03.336296 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.336289 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-fsbvq\"" Jul 05 11:54:03.336479 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.336338 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 05 11:54:03.351383 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.351354 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:03.402619 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.402590 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:03.477359 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.477327 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99q2t\" (UniqueName: \"kubernetes.io/projected/87856c3e-d6be-483e-b830-7b13000c87c5-kube-api-access-99q2t\") pod \"migrator-d57d58cc9-kt5dh\" (UID: \"87856c3e-d6be-483e-b830-7b13000c87c5\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh" Jul 05 11:54:03.578029 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.578002 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99q2t\" (UniqueName: \"kubernetes.io/projected/87856c3e-d6be-483e-b830-7b13000c87c5-kube-api-access-99q2t\") pod \"migrator-d57d58cc9-kt5dh\" (UID: \"87856c3e-d6be-483e-b830-7b13000c87c5\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh" Jul 05 11:54:03.585800 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.585780 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99q2t\" (UniqueName: \"kubernetes.io/projected/87856c3e-d6be-483e-b830-7b13000c87c5-kube-api-access-99q2t\") pod \"migrator-d57d58cc9-kt5dh\" (UID: \"87856c3e-d6be-483e-b830-7b13000c87c5\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh" Jul 05 11:54:03.643589 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.642491 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh" Jul 05 11:54:03.886545 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:03.886433 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh"] Jul 05 11:54:03.897035 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:54:03.896999 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87856c3e_d6be_483e_b830_7b13000c87c5.slice/crio-7f7b7b1043c45c3034dff2b22f07398dfedcdf7b8fc53099f5fe10216778227d WatchSource:0}: Error finding container 7f7b7b1043c45c3034dff2b22f07398dfedcdf7b8fc53099f5fe10216778227d: Status 404 returned error can't find the container with id 7f7b7b1043c45c3034dff2b22f07398dfedcdf7b8fc53099f5fe10216778227d Jul 05 11:54:04.176403 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.176311 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:04.278095 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.278055 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:04.329801 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.329760 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:04.498672 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.498568 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" event={"ID":"7197549e-9314-42dd-bbdb-d53feb6ece0e","Type":"ContainerStarted","Data":"7b0f78937de7d9bbbf6630722f2b67779fc4b0abd099289d3f94256c56d15d5a"} Jul 05 11:54:04.499773 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.499746 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh" event={"ID":"87856c3e-d6be-483e-b830-7b13000c87c5","Type":"ContainerStarted","Data":"7f7b7b1043c45c3034dff2b22f07398dfedcdf7b8fc53099f5fe10216778227d"} Jul 05 11:54:04.505890 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.505844 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-768g2" event={"ID":"02ab04f0-b13c-4ee6-8646-715089510936","Type":"ContainerStarted","Data":"dd32c686817958df13163e0eede7fd925e85a7f47814b340e4ba8585c6cef21c"} Jul 05 11:54:04.509336 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.509309 2575 generic.go:356] "Generic (PLEG): container finished" podID="43726046-5603-4270-94d8-4e3f27f984b7" containerID="1cf2d38d655f5d7624921b6dbef78d7727d3322fd7174f8f4ce7c193ebad8a0e" exitCode=0 Jul 05 11:54:04.509486 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.509392 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bc4j" event={"ID":"43726046-5603-4270-94d8-4e3f27f984b7","Type":"ContainerDied","Data":"1cf2d38d655f5d7624921b6dbef78d7727d3322fd7174f8f4ce7c193ebad8a0e"} Jul 05 11:54:04.511107 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.511079 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f67t2" event={"ID":"4b444ec4-9b23-422d-ac9d-dd3535492d17","Type":"ContainerStarted","Data":"9320c3794b1f511d77979080b0541d85ce4b2cfdd5d80ee6eadbfb465f00a764"} Jul 05 11:54:04.512929 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.512906 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8z5fg" event={"ID":"746ca75a-b87d-4153-9e6a-74845453758c","Type":"ContainerStarted","Data":"8f8ae40d7452f6d6d2134a75247b12ac669b0128a21525aff1d8a00110ec1878"} Jul 05 11:54:04.514885 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.514848 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/0.log" Jul 05 11:54:04.514990 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.514911 2575 generic.go:356] "Generic (PLEG): container finished" podID="5f7f19ec-9537-4581-b1ad-62d17debda03" containerID="5e52c2d47a68f22d52e52c10b1c43dc3eeccc30be9426d188c00faa0b15b246d" exitCode=255 Jul 05 11:54:04.514990 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.514980 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" event={"ID":"5f7f19ec-9537-4581-b1ad-62d17debda03","Type":"ContainerDied","Data":"5e52c2d47a68f22d52e52c10b1c43dc3eeccc30be9426d188c00faa0b15b246d"} Jul 05 11:54:04.515169 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.515151 2575 scope.go:117] "RemoveContainer" containerID="5e52c2d47a68f22d52e52c10b1c43dc3eeccc30be9426d188c00faa0b15b246d" Jul 05 11:54:04.516992 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.516953 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" event={"ID":"1881b431-0b27-4e0f-a16f-6a0e93cafaa3","Type":"ContainerStarted","Data":"fb2f1b9afc6449cd519c7b2caf45b3ffe52f2fa79b8a20417057d7e62ee346dc"} Jul 05 11:54:04.519344 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.519321 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rt4k7" event={"ID":"3997821f-4556-407b-9b73-daf87e87f9b1","Type":"ContainerStarted","Data":"e663c05e8c318295e97b0eaf9445d220af8dcc20e1a1779644c8d09121d93eb8"} Jul 05 11:54:04.531712 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.531659 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" podStartSLOduration=26.095002217 podStartE2EDuration="36.531643888s" totalImagesPullingTime="10.436641671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:04.530566743 +0000 UTC m=+44.921268506" watchObservedRunningTime="2026-07-05 11:54:04.531643888 +0000 UTC m=+44.922345650" Jul 05 11:54:04.533261 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.533222 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:54:04.550702 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.550059 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" podStartSLOduration=26.036885088 podStartE2EDuration="36.550038318s" totalImagesPullingTime="10.51315323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:04.548858126 +0000 UTC m=+44.939559890" watchObservedRunningTime="2026-07-05 11:54:04.550038318 +0000 UTC m=+44.940740081" Jul 05 11:54:04.599725 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.599498 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-f67t2" podStartSLOduration=4.194851368 podStartE2EDuration="14.599477082s" totalImagesPullingTime="10.404625714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:04.572474326 +0000 UTC m=+44.963176091" watchObservedRunningTime="2026-07-05 11:54:04.599477082 +0000 UTC m=+44.990178845" Jul 05 11:54:04.636722 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.636111 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8z5fg" podStartSLOduration=33.361712216 podStartE2EDuration="43.636089806s" totalImagesPullingTime="10.27437759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:04.635043928 +0000 UTC m=+45.025745684" watchObservedRunningTime="2026-07-05 11:54:04.636089806 +0000 UTC m=+45.026791567" Jul 05 11:54:04.676248 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.674694 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rt4k7" podStartSLOduration=34.432941129 podStartE2EDuration="44.674672348s" totalImagesPullingTime="10.241731219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:04.674421939 +0000 UTC m=+45.065123704" watchObservedRunningTime="2026-07-05 11:54:04.674672348 +0000 UTC m=+45.065374112" Jul 05 11:54:04.676248 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.675564 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-768g2" podStartSLOduration=11.247294701 podStartE2EDuration="21.675546087s" totalImagesPullingTime="10.428251386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:04.651204651 +0000 UTC m=+45.041906413" watchObservedRunningTime="2026-07-05 11:54:04.675546087 +0000 UTC m=+45.066247851" Jul 05 11:54:04.817185 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.817146 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:04.870307 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:04.870275 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:05.017944 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.017905 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:05.117743 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.117703 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:05.254601 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.254516 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:05.355470 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.355432 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:05.518730 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.518637 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:05.525550 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.525522 2575 generic.go:356] "Generic (PLEG): container finished" podID="43726046-5603-4270-94d8-4e3f27f984b7" containerID="fe9394816acd66348244c7e2f7a905c01d4c17e89627ddad4f35c4a4b2c74c6e" exitCode=0 Jul 05 11:54:05.525694 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.525604 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bc4j" event={"ID":"43726046-5603-4270-94d8-4e3f27f984b7","Type":"ContainerDied","Data":"fe9394816acd66348244c7e2f7a905c01d4c17e89627ddad4f35c4a4b2c74c6e"} Jul 05 11:54:05.527293 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.527270 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 11:54:05.527661 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.527646 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/0.log" Jul 05 11:54:05.527709 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.527676 2575 generic.go:356] "Generic (PLEG): container finished" podID="5f7f19ec-9537-4581-b1ad-62d17debda03" containerID="40e3d8544b74ad88c4904de7439937d42b4210d0a15c6564106cd609bc4dd092" exitCode=255 Jul 05 11:54:05.527748 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.527729 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" event={"ID":"5f7f19ec-9537-4581-b1ad-62d17debda03","Type":"ContainerDied","Data":"40e3d8544b74ad88c4904de7439937d42b4210d0a15c6564106cd609bc4dd092"} Jul 05 11:54:05.527798 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.527766 2575 scope.go:117] "RemoveContainer" containerID="5e52c2d47a68f22d52e52c10b1c43dc3eeccc30be9426d188c00faa0b15b246d" Jul 05 11:54:05.528202 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.527943 2575 scope.go:117] "RemoveContainer" containerID="40e3d8544b74ad88c4904de7439937d42b4210d0a15c6564106cd609bc4dd092" Jul 05 11:54:05.528202 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:05.528089 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-4sth7_openshift-console-operator(5f7f19ec-9537-4581-b1ad-62d17debda03)\"" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" podUID="5f7f19ec-9537-4581-b1ad-62d17debda03" Jul 05 11:54:05.577261 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.577232 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:05.860115 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.860070 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m7tqb"] Jul 05 11:54:05.884342 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.884307 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m7tqb"] Jul 05 11:54:05.884513 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.884431 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:05.887096 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.887070 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 05 11:54:05.887234 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.887133 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bxxhf\"" Jul 05 11:54:05.887306 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.887229 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 05 11:54:05.903544 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:05.903519 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:06.005710 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.005671 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bw9c\" (UniqueName: \"kubernetes.io/projected/42dcc512-1104-4580-a06a-42db9e8317f7-kube-api-access-4bw9c\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.005929 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.005732 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42dcc512-1104-4580-a06a-42db9e8317f7-data-volume\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.005929 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.005751 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.005929 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.005836 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42dcc512-1104-4580-a06a-42db9e8317f7-crio-socket\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.006077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.005928 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.036568 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.036528 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:06.107005 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.106977 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bw9c\" (UniqueName: \"kubernetes.io/projected/42dcc512-1104-4580-a06a-42db9e8317f7-kube-api-access-4bw9c\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.107122 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.107039 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42dcc512-1104-4580-a06a-42db9e8317f7-data-volume\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.107122 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.107061 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.107122 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.107082 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42dcc512-1104-4580-a06a-42db9e8317f7-crio-socket\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.107122 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.107112 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.107306 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.107181 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 11:54:06.107306 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.107217 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42dcc512-1104-4580-a06a-42db9e8317f7-crio-socket\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.107306 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.107242 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 11:54:06.107306 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.107246 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm podName:42dcc512-1104-4580-a06a-42db9e8317f7 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:06.607227653 +0000 UTC m=+46.997929404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m7tqb" (UID: "42dcc512-1104-4580-a06a-42db9e8317f7") : configmap "kube-rbac-proxy" not found Jul 05 11:54:06.107306 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.107276 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls podName:42dcc512-1104-4580-a06a-42db9e8317f7 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:06.607265808 +0000 UTC m=+46.997967569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m7tqb" (UID: "42dcc512-1104-4580-a06a-42db9e8317f7") : secret "insights-runtime-extractor-tls" not found Jul 05 11:54:06.107489 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.107341 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42dcc512-1104-4580-a06a-42db9e8317f7-data-volume\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.115456 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.115400 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bw9c\" (UniqueName: \"kubernetes.io/projected/42dcc512-1104-4580-a06a-42db9e8317f7-kube-api-access-4bw9c\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.138802 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.138756 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:06.308513 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.308476 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:54:06.308699 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.308533 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:54:06.308699 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.308642 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 11:54:06.308699 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.308646 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 11:54:06.308699 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.308667 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-96db66bd7-drwx2: secret "image-registry-tls" not found Jul 05 11:54:06.308699 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.308694 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls podName:8efd845e-7e3b-4a9f-9572-43d72a6a22f0 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:22.308681665 +0000 UTC m=+62.699383405 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-479rr" (UID: "8efd845e-7e3b-4a9f-9572-43d72a6a22f0") : secret "samples-operator-tls" not found Jul 05 11:54:06.309011 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.308706 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls podName:05dde2fd-fba0-412f-90a3-97cc45e9c537 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:22.308700924 +0000 UTC m=+62.699402665 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls") pod "image-registry-96db66bd7-drwx2" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537") : secret "image-registry-tls" not found Jul 05 11:54:06.409414 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.409380 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:54:06.409573 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.409500 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 11:54:06.409573 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.409550 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls podName:31952da1-f50a-4817-b342-775273f214f5 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:22.40953678 +0000 UTC m=+62.800238521 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls") pod "dns-default-gplkw" (UID: "31952da1-f50a-4817-b342-775273f214f5") : secret "dns-default-metrics-tls" not found Jul 05 11:54:06.532705 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.532668 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh" event={"ID":"87856c3e-d6be-483e-b830-7b13000c87c5","Type":"ContainerStarted","Data":"e072965032c54406c8407eadef31b16a6ff8804b614c07fbe65fd4d70825fd7d"} Jul 05 11:54:06.532913 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.532729 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh" event={"ID":"87856c3e-d6be-483e-b830-7b13000c87c5","Type":"ContainerStarted","Data":"33e381aa63f7c7c70b1850bcf43eff3bbc6d32b699e7dc1ce40f3fe70b7040c3"} Jul 05 11:54:06.535775 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.535751 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bc4j" event={"ID":"43726046-5603-4270-94d8-4e3f27f984b7","Type":"ContainerStarted","Data":"2304dfd40713cfe640361c788187ada00152f55ccda40fc87f1f740e51d369d8"} Jul 05 11:54:06.537251 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.537233 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 11:54:06.537488 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.537474 2575 scope.go:117] "RemoveContainer" containerID="40e3d8544b74ad88c4904de7439937d42b4210d0a15c6564106cd609bc4dd092" Jul 05 11:54:06.537640 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.537623 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-4sth7_openshift-console-operator(5f7f19ec-9537-4581-b1ad-62d17debda03)\"" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" podUID="5f7f19ec-9537-4581-b1ad-62d17debda03" Jul 05 11:54:06.549133 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.549093 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kt5dh" podStartSLOduration=1.5428501510000001 podStartE2EDuration="3.549079819s" totalImagesPullingTime="2.006229668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:06.5486538 +0000 UTC m=+46.939355584" watchObservedRunningTime="2026-07-05 11:54:06.549079819 +0000 UTC m=+46.939781594" Jul 05 11:54:06.570031 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.569995 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6bc4j" podStartSLOduration=6.923540805 podStartE2EDuration="45.569984145s" totalImagesPullingTime="38.64644334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:06.569310446 +0000 UTC m=+46.960012210" watchObservedRunningTime="2026-07-05 11:54:06.569984145 +0000 UTC m=+46.960685906" Jul 05 11:54:06.611167 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.611127 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.611339 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.611198 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:06.611339 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.611250 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 11:54:06.611339 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.611310 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm podName:42dcc512-1104-4580-a06a-42db9e8317f7 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:07.611296221 +0000 UTC m=+48.001997960 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m7tqb" (UID: "42dcc512-1104-4580-a06a-42db9e8317f7") : configmap "kube-rbac-proxy" not found Jul 05 11:54:06.611339 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.611331 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 11:54:06.611521 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:06.611383 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls podName:42dcc512-1104-4580-a06a-42db9e8317f7 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:07.611370972 +0000 UTC m=+48.002072711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m7tqb" (UID: "42dcc512-1104-4580-a06a-42db9e8317f7") : secret "insights-runtime-extractor-tls" not found Jul 05 11:54:06.689780 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.689686 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:06.751290 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.751254 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:06.960354 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:06.960266 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:07.114472 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.114437 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-tnvhz"] Jul 05 11:54:07.139689 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.139661 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-tnvhz"] Jul 05 11:54:07.139827 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.139784 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-tnvhz" Jul 05 11:54:07.142730 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.142701 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-48fw9\"" Jul 05 11:54:07.142886 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.142743 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 05 11:54:07.142886 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.142701 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 05 11:54:07.143721 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.143703 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 05 11:54:07.143857 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.143843 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 05 11:54:07.145424 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.145401 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:07.245936 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.245810 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:07.315940 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.315908 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzjc2\" (UniqueName: \"kubernetes.io/projected/899c839d-c5d9-4e75-a18e-1d75d0290300-kube-api-access-pzjc2\") pod \"service-ca-758f599c5-tnvhz\" (UID: \"899c839d-c5d9-4e75-a18e-1d75d0290300\") " pod="openshift-service-ca/service-ca-758f599c5-tnvhz" Jul 05 11:54:07.315940 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.315946 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/899c839d-c5d9-4e75-a18e-1d75d0290300-signing-key\") pod \"service-ca-758f599c5-tnvhz\" (UID: \"899c839d-c5d9-4e75-a18e-1d75d0290300\") " pod="openshift-service-ca/service-ca-758f599c5-tnvhz" Jul 05 11:54:07.316155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.316013 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/899c839d-c5d9-4e75-a18e-1d75d0290300-signing-cabundle\") pod \"service-ca-758f599c5-tnvhz\" (UID: \"899c839d-c5d9-4e75-a18e-1d75d0290300\") " pod="openshift-service-ca/service-ca-758f599c5-tnvhz" Jul 05 11:54:07.417023 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.416984 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzjc2\" (UniqueName: \"kubernetes.io/projected/899c839d-c5d9-4e75-a18e-1d75d0290300-kube-api-access-pzjc2\") pod \"service-ca-758f599c5-tnvhz\" (UID: \"899c839d-c5d9-4e75-a18e-1d75d0290300\") " pod="openshift-service-ca/service-ca-758f599c5-tnvhz" Jul 05 11:54:07.417023 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.417024 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/899c839d-c5d9-4e75-a18e-1d75d0290300-signing-key\") pod \"service-ca-758f599c5-tnvhz\" (UID: \"899c839d-c5d9-4e75-a18e-1d75d0290300\") " pod="openshift-service-ca/service-ca-758f599c5-tnvhz" Jul 05 11:54:07.417211 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.417072 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/899c839d-c5d9-4e75-a18e-1d75d0290300-signing-cabundle\") pod \"service-ca-758f599c5-tnvhz\" (UID: \"899c839d-c5d9-4e75-a18e-1d75d0290300\") " pod="openshift-service-ca/service-ca-758f599c5-tnvhz" Jul 05 11:54:07.417890 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.417848 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/899c839d-c5d9-4e75-a18e-1d75d0290300-signing-cabundle\") pod \"service-ca-758f599c5-tnvhz\" (UID: \"899c839d-c5d9-4e75-a18e-1d75d0290300\") " pod="openshift-service-ca/service-ca-758f599c5-tnvhz" Jul 05 11:54:07.419544 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.419518 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/899c839d-c5d9-4e75-a18e-1d75d0290300-signing-key\") pod \"service-ca-758f599c5-tnvhz\" (UID: \"899c839d-c5d9-4e75-a18e-1d75d0290300\") " pod="openshift-service-ca/service-ca-758f599c5-tnvhz" Jul 05 11:54:07.425629 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.425610 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzjc2\" (UniqueName: \"kubernetes.io/projected/899c839d-c5d9-4e75-a18e-1d75d0290300-kube-api-access-pzjc2\") pod \"service-ca-758f599c5-tnvhz\" (UID: \"899c839d-c5d9-4e75-a18e-1d75d0290300\") " pod="openshift-service-ca/service-ca-758f599c5-tnvhz" Jul 05 11:54:07.449607 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.449583 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-tnvhz" Jul 05 11:54:07.589833 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.589796 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-tnvhz"] Jul 05 11:54:07.592845 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:54:07.592816 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod899c839d_c5d9_4e75_a18e_1d75d0290300.slice/crio-5259f9c729c25a8ac8fc28b7270e24c53c1fc5c925e64a40783e0f970ab69e70 WatchSource:0}: Error finding container 5259f9c729c25a8ac8fc28b7270e24c53c1fc5c925e64a40783e0f970ab69e70: Status 404 returned error can't find the container with id 5259f9c729c25a8ac8fc28b7270e24c53c1fc5c925e64a40783e0f970ab69e70 Jul 05 11:54:07.618861 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.618833 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:07.618994 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.618942 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:07.618994 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:07.618974 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 11:54:07.619063 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:07.619011 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 11:54:07.619063 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:07.619025 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls podName:42dcc512-1104-4580-a06a-42db9e8317f7 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:09.619009441 +0000 UTC m=+50.009711181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m7tqb" (UID: "42dcc512-1104-4580-a06a-42db9e8317f7") : secret "insights-runtime-extractor-tls" not found Jul 05 11:54:07.619063 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:07.619045 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm podName:42dcc512-1104-4580-a06a-42db9e8317f7 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:09.619035003 +0000 UTC m=+50.009736770 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m7tqb" (UID: "42dcc512-1104-4580-a06a-42db9e8317f7") : configmap "kube-rbac-proxy" not found Jul 05 11:54:07.984557 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:07.984519 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:08.057285 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:08.057255 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hb68f_9930e6e9-d5bc-4384-a9de-97740b2b692d/dns-node-resolver/0.log" Jul 05 11:54:08.549709 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:08.549674 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-tnvhz" event={"ID":"899c839d-c5d9-4e75-a18e-1d75d0290300","Type":"ContainerStarted","Data":"6dfa8a85f52aa40a418b83750c2ff4523caa7f958340563abfd4fc70ba3f37b2"} Jul 05 11:54:08.549885 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:08.549723 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-tnvhz" event={"ID":"899c839d-c5d9-4e75-a18e-1d75d0290300","Type":"ContainerStarted","Data":"5259f9c729c25a8ac8fc28b7270e24c53c1fc5c925e64a40783e0f970ab69e70"} Jul 05 11:54:08.566005 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:08.565959 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-tnvhz" podStartSLOduration=1.565944289 podStartE2EDuration="1.565944289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:08.565833814 +0000 UTC m=+48.956535577" watchObservedRunningTime="2026-07-05 11:54:08.565944289 +0000 UTC m=+48.956646051" Jul 05 11:54:08.658917 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:08.658861 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s2dpw_afa4b888-1bc8-4865-911e-5720c5b0028a/node-ca/0.log" Jul 05 11:54:08.777419 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:08.777384 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:08.971046 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:08.971002 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:09.057259 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:09.057163 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f67t2_4b444ec4-9b23-422d-ac9d-dd3535492d17/serve-healthcheck-canary/0.log" Jul 05 11:54:09.077733 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:09.077698 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:09.255324 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:09.255238 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:09.255720 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:09.255695 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kt5dh_87856c3e-d6be-483e-b830-7b13000c87c5/migrator/0.log" Jul 05 11:54:09.356361 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:09.356325 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:09.456261 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:09.456234 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kt5dh_87856c3e-d6be-483e-b830-7b13000c87c5/graceful-termination/0.log" Jul 05 11:54:09.456970 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:09.456936 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:09.636467 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:09.636434 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:09.636630 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:09.636484 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:09.636630 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:09.636563 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 11:54:09.636630 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:09.636576 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 11:54:09.636630 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:09.636629 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls podName:42dcc512-1104-4580-a06a-42db9e8317f7 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:13.636616588 +0000 UTC m=+54.027318327 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m7tqb" (UID: "42dcc512-1104-4580-a06a-42db9e8317f7") : secret "insights-runtime-extractor-tls" not found Jul 05 11:54:09.636762 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:09.636644 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm podName:42dcc512-1104-4580-a06a-42db9e8317f7 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:13.636638429 +0000 UTC m=+54.027340168 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m7tqb" (UID: "42dcc512-1104-4580-a06a-42db9e8317f7") : configmap "kube-rbac-proxy" not found Jul 05 11:54:09.658268 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:09.658234 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-j8f2q_be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34/kube-storage-version-migrator-operator/0.log" Jul 05 11:54:10.461362 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:10.461322 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:10.660270 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:10.660235 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:10.833348 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:10.833318 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:54:10.833532 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:10.833381 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:54:10.833739 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:10.833727 2575 scope.go:117] "RemoveContainer" containerID="40e3d8544b74ad88c4904de7439937d42b4210d0a15c6564106cd609bc4dd092" Jul 05 11:54:10.833929 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:10.833910 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-4sth7_openshift-console-operator(5f7f19ec-9537-4581-b1ad-62d17debda03)\"" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" podUID="5f7f19ec-9537-4581-b1ad-62d17debda03" Jul 05 11:54:10.969928 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:10.969894 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:11.069310 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:11.069277 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:11.259092 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:11.259006 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:11.670551 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:11.670515 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:12.261130 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:12.261094 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:12.461736 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:12.461696 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:12.561928 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:12.561893 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:12.859484 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:12.859386 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:13.059404 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:13.059361 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:13.256595 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:13.256483 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:13.304821 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:13.304788 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:13.658305 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:13.658270 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:13.670120 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:13.670088 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:13.670247 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:13.670171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:13.670247 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:13.670239 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 11:54:13.670329 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:13.670243 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 11:54:13.670329 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:13.670284 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm podName:42dcc512-1104-4580-a06a-42db9e8317f7 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:21.670271445 +0000 UTC m=+62.060973185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m7tqb" (UID: "42dcc512-1104-4580-a06a-42db9e8317f7") : configmap "kube-rbac-proxy" not found Jul 05 11:54:13.670329 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:54:13.670317 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls podName:42dcc512-1104-4580-a06a-42db9e8317f7 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:21.670297938 +0000 UTC m=+62.060999685 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m7tqb" (UID: "42dcc512-1104-4580-a06a-42db9e8317f7") : secret "insights-runtime-extractor-tls" not found Jul 05 11:54:14.064056 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:14.064017 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:14.264032 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:14.263997 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:14.664003 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:14.663969 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:15.046335 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:15.046251 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:15.149008 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:15.148967 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:15.424083 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:15.424054 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngxcf" Jul 05 11:54:15.465545 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:15.465505 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:15.655409 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:15.655371 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:15.755821 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:15.755723 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:15.856658 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:15.856611 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:16.058008 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:16.057977 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:16.661155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:16.661120 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:17.055186 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:17.055141 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:17.457090 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:17.456975 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:17.856296 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:17.856262 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:18.262179 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:18.262101 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:18.458656 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:18.458618 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:18.857202 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:18.857167 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:19.061161 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:19.061131 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:21.333204 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:21.333156 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:21.550930 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:21.550869 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:21.735689 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:21.735596 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:21.735689 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:21.735663 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:21.736202 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:21.736175 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42dcc512-1104-4580-a06a-42db9e8317f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:21.738014 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:21.737990 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42dcc512-1104-4580-a06a-42db9e8317f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7tqb\" (UID: \"42dcc512-1104-4580-a06a-42db9e8317f7\") " pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:21.795649 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:21.795617 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bxxhf\"" Jul 05 11:54:21.803804 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:21.803781 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m7tqb" Jul 05 11:54:21.967277 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:21.967243 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m7tqb"] Jul 05 11:54:21.970196 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:54:21.970168 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42dcc512_1104_4580_a06a_42db9e8317f7.slice/crio-80791c05159103c0ce11ca3caf70a8b5e9e5e5937c0dfc415f88476231198c90 WatchSource:0}: Error finding container 80791c05159103c0ce11ca3caf70a8b5e9e5e5937c0dfc415f88476231198c90: Status 404 returned error can't find the container with id 80791c05159103c0ce11ca3caf70a8b5e9e5e5937c0dfc415f88476231198c90 Jul 05 11:54:22.341925 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.341891 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:54:22.342379 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.341970 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:54:22.344494 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.344471 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8efd845e-7e3b-4a9f-9572-43d72a6a22f0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-479rr\" (UID: \"8efd845e-7e3b-4a9f-9572-43d72a6a22f0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:54:22.344567 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.344471 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls\") pod \"image-registry-96db66bd7-drwx2\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:54:22.443239 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.443202 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:54:22.445499 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.445478 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31952da1-f50a-4817-b342-775273f214f5-metrics-tls\") pod \"dns-default-gplkw\" (UID: \"31952da1-f50a-4817-b342-775273f214f5\") " pod="openshift-dns/dns-default-gplkw" Jul 05 11:54:22.551584 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.551544 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mbvkr\"" Jul 05 11:54:22.559822 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.559753 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:54:22.595464 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.595324 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zmtrj\"" Jul 05 11:54:22.596347 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.596305 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m7tqb" event={"ID":"42dcc512-1104-4580-a06a-42db9e8317f7","Type":"ContainerStarted","Data":"80791c05159103c0ce11ca3caf70a8b5e9e5e5937c0dfc415f88476231198c90"} Jul 05 11:54:22.602420 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.602395 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" Jul 05 11:54:22.696694 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.695907 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gbwbs\"" Jul 05 11:54:22.702288 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.702257 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gplkw" Jul 05 11:54:22.759960 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.759517 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-96db66bd7-drwx2"] Jul 05 11:54:22.761707 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:54:22.761678 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05dde2fd_fba0_412f_90a3_97cc45e9c537.slice/crio-cbed1868fa2c316f25763be5c939f86119d7210fa53598c37dd4bfe2b15a1ac5 WatchSource:0}: Error finding container cbed1868fa2c316f25763be5c939f86119d7210fa53598c37dd4bfe2b15a1ac5: Status 404 returned error can't find the container with id cbed1868fa2c316f25763be5c939f86119d7210fa53598c37dd4bfe2b15a1ac5 Jul 05 11:54:22.791146 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.790804 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr"] Jul 05 11:54:22.912124 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:22.911927 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gplkw"] Jul 05 11:54:23.602370 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:23.602294 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gplkw" event={"ID":"31952da1-f50a-4817-b342-775273f214f5","Type":"ContainerStarted","Data":"37ff3f4f9e124eff26a9709aeebb247659d1c236059d5a8d05d8df6319b5061c"} Jul 05 11:54:23.603890 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:23.603848 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" event={"ID":"8efd845e-7e3b-4a9f-9572-43d72a6a22f0","Type":"ContainerStarted","Data":"363784bd502d6fbf5d5d8d6f964795bcd13c2c6bdfffd3ff4858b0834bbe176c"} Jul 05 11:54:23.605654 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:23.605629 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" event={"ID":"05dde2fd-fba0-412f-90a3-97cc45e9c537","Type":"ContainerStarted","Data":"abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b"} Jul 05 11:54:23.605766 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:23.605672 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" event={"ID":"05dde2fd-fba0-412f-90a3-97cc45e9c537","Type":"ContainerStarted","Data":"cbed1868fa2c316f25763be5c939f86119d7210fa53598c37dd4bfe2b15a1ac5"} Jul 05 11:54:23.605825 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:23.605781 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:54:23.629839 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:23.629783 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" podStartSLOduration=62.629763917 podStartE2EDuration="1m2.629763917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:23.627223832 +0000 UTC m=+64.017925594" watchObservedRunningTime="2026-07-05 11:54:23.629763917 +0000 UTC m=+64.020465680" Jul 05 11:54:24.962075 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:24.962028 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:54:24.964932 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:24.964900 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793d5b93-3b3a-4f7b-99fb-38db6730bfa6-metrics-certs\") pod \"network-metrics-daemon-dhzh9\" (UID: \"793d5b93-3b3a-4f7b-99fb-38db6730bfa6\") " pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:54:25.124164 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:25.124129 2575 scope.go:117] "RemoveContainer" containerID="40e3d8544b74ad88c4904de7439937d42b4210d0a15c6564106cd609bc4dd092" Jul 05 11:54:25.163856 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:25.163831 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-759jc\"" Jul 05 11:54:25.172013 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:25.171988 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dhzh9" Jul 05 11:54:26.070384 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.070341 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm"] Jul 05 11:54:26.087705 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.087672 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg"] Jul 05 11:54:26.087894 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.087792 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm" Jul 05 11:54:26.094124 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.094095 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 05 11:54:26.094519 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.094488 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-scb9b\"" Jul 05 11:54:26.094637 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.094578 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 05 11:54:26.102757 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.102725 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x"] Jul 05 11:54:26.102996 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.102935 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" Jul 05 11:54:26.108523 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.108465 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rzrgq\"" Jul 05 11:54:26.110953 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.110932 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 05 11:54:26.110953 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.110944 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 05 11:54:26.112794 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.112773 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 05 11:54:26.112913 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.112776 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 05 11:54:26.138102 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.138070 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:26.141312 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.141278 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 05 11:54:26.141555 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.141535 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 05 11:54:26.141661 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.141592 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 05 11:54:26.141914 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.141893 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 05 11:54:26.143469 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.143450 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k"] Jul 05 11:54:26.170525 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.170439 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxpz5\" (UniqueName: \"kubernetes.io/projected/e13f32f9-100f-456f-bb39-1dac54b64af9-kube-api-access-vxpz5\") pod \"cluster-monitoring-operator-64786d9cbb-ckggg\" (UID: \"e13f32f9-100f-456f-bb39-1dac54b64af9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" Jul 05 11:54:26.170525 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.170479 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vw5m\" (UniqueName: \"kubernetes.io/projected/6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568-kube-api-access-8vw5m\") pod \"klusterlet-addon-workmgr-55ff98c4d7-ppr5x\" (UID: \"6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:26.170525 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.170511 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568-klusterlet-config\") pod \"klusterlet-addon-workmgr-55ff98c4d7-ppr5x\" (UID: \"6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:26.170806 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.170535 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e13f32f9-100f-456f-bb39-1dac54b64af9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ckggg\" (UID: \"e13f32f9-100f-456f-bb39-1dac54b64af9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" Jul 05 11:54:26.170806 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.170591 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568-tmp\") pod \"klusterlet-addon-workmgr-55ff98c4d7-ppr5x\" (UID: \"6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:26.170806 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.170634 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e13f32f9-100f-456f-bb39-1dac54b64af9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ckggg\" (UID: \"e13f32f9-100f-456f-bb39-1dac54b64af9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" Jul 05 11:54:26.170806 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.170689 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c843d931-14bd-431f-9263-9671ea3401a8-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-8pfwm\" (UID: \"c843d931-14bd-431f-9263-9671ea3401a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm" Jul 05 11:54:26.170806 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.170761 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c843d931-14bd-431f-9263-9671ea3401a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8pfwm\" (UID: \"c843d931-14bd-431f-9263-9671ea3401a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm" Jul 05 11:54:26.171929 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.171904 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-77bbc49f9c-hdjwj"] Jul 05 11:54:26.172077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.172055 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.175077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.175048 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 05 11:54:26.175077 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.175063 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 05 11:54:26.175243 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.175145 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 05 11:54:26.175243 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.175162 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 05 11:54:26.180947 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.180851 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:26.189395 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.189373 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg"] Jul 05 11:54:26.189486 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.189405 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm"] Jul 05 11:54:26.189486 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.189419 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x"] Jul 05 11:54:26.189486 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.189436 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh"] Jul 05 11:54:26.189486 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.189476 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.194927 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.194907 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 05 11:54:26.195030 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.194949 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 05 11:54:26.195030 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.194958 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8dsnd\"" Jul 05 11:54:26.195030 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.194987 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 05 11:54:26.195198 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.194963 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 05 11:54:26.195457 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.195407 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 05 11:54:26.195765 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.195737 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 05 11:54:26.211668 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.211646 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77bbc49f9c-hdjwj"] Jul 05 11:54:26.211809 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.211686 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k"] Jul 05 11:54:26.211809 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.211712 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh"] Jul 05 11:54:26.211809 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.211798 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh" Jul 05 11:54:26.214410 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.214390 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-ktngk\"" Jul 05 11:54:26.214666 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.214477 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 05 11:54:26.255937 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.255902 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:26.271249 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271216 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxpz5\" (UniqueName: \"kubernetes.io/projected/e13f32f9-100f-456f-bb39-1dac54b64af9-kube-api-access-vxpz5\") pod \"cluster-monitoring-operator-64786d9cbb-ckggg\" (UID: \"e13f32f9-100f-456f-bb39-1dac54b64af9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" Jul 05 11:54:26.271417 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271360 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vw5m\" (UniqueName: \"kubernetes.io/projected/6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568-kube-api-access-8vw5m\") pod \"klusterlet-addon-workmgr-55ff98c4d7-ppr5x\" (UID: \"6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:26.271417 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271391 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8hxm\" (UniqueName: \"kubernetes.io/projected/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-kube-api-access-c8hxm\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.271491 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271423 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568-klusterlet-config\") pod \"klusterlet-addon-workmgr-55ff98c4d7-ppr5x\" (UID: \"6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:26.271537 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271481 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e13f32f9-100f-456f-bb39-1dac54b64af9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ckggg\" (UID: \"e13f32f9-100f-456f-bb39-1dac54b64af9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" Jul 05 11:54:26.271586 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271534 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-ca\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.271586 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271567 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.271675 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271628 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h9nz\" (UniqueName: \"kubernetes.io/projected/c635fb3b-5ece-42e4-a97b-364b7c7f99ee-kube-api-access-5h9nz\") pod \"managed-serviceaccount-addon-agent-b8cd87856-nxzjh\" (UID: \"c635fb3b-5ece-42e4-a97b-364b7c7f99ee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh" Jul 05 11:54:26.271675 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271662 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/842db65b-2a1c-4d56-bf76-72b3115a18f0-stats-auth\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.271773 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271695 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.271773 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271726 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568-tmp\") pod \"klusterlet-addon-workmgr-55ff98c4d7-ppr5x\" (UID: \"6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:26.271773 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271749 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c635fb3b-5ece-42e4-a97b-364b7c7f99ee-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b8cd87856-nxzjh\" (UID: \"c635fb3b-5ece-42e4-a97b-364b7c7f99ee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh" Jul 05 11:54:26.271935 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271784 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhcgb\" (UniqueName: \"kubernetes.io/projected/842db65b-2a1c-4d56-bf76-72b3115a18f0-kube-api-access-zhcgb\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.271935 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271827 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e13f32f9-100f-456f-bb39-1dac54b64af9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ckggg\" (UID: \"e13f32f9-100f-456f-bb39-1dac54b64af9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" Jul 05 11:54:26.271935 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271866 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c843d931-14bd-431f-9263-9671ea3401a8-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-8pfwm\" (UID: \"c843d931-14bd-431f-9263-9671ea3401a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm" Jul 05 11:54:26.271935 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271913 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c843d931-14bd-431f-9263-9671ea3401a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8pfwm\" (UID: \"c843d931-14bd-431f-9263-9671ea3401a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm" Jul 05 11:54:26.272143 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271937 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-hub\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.272143 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271961 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842db65b-2a1c-4d56-bf76-72b3115a18f0-service-ca-bundle\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.272143 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.271989 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/842db65b-2a1c-4d56-bf76-72b3115a18f0-default-certificate\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.272143 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.272028 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.272143 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.272053 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842db65b-2a1c-4d56-bf76-72b3115a18f0-metrics-certs\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.272403 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.272160 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568-tmp\") pod \"klusterlet-addon-workmgr-55ff98c4d7-ppr5x\" (UID: \"6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:26.272403 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.272329 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e13f32f9-100f-456f-bb39-1dac54b64af9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ckggg\" (UID: \"e13f32f9-100f-456f-bb39-1dac54b64af9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" Jul 05 11:54:26.272832 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.272807 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c843d931-14bd-431f-9263-9671ea3401a8-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-8pfwm\" (UID: \"c843d931-14bd-431f-9263-9671ea3401a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm" Jul 05 11:54:26.274772 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.274740 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568-klusterlet-config\") pod \"klusterlet-addon-workmgr-55ff98c4d7-ppr5x\" (UID: \"6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:26.274919 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.274795 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e13f32f9-100f-456f-bb39-1dac54b64af9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ckggg\" (UID: \"e13f32f9-100f-456f-bb39-1dac54b64af9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" Jul 05 11:54:26.274997 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.274963 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c843d931-14bd-431f-9263-9671ea3401a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8pfwm\" (UID: \"c843d931-14bd-431f-9263-9671ea3401a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm" Jul 05 11:54:26.285452 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.285427 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxpz5\" (UniqueName: \"kubernetes.io/projected/e13f32f9-100f-456f-bb39-1dac54b64af9-kube-api-access-vxpz5\") pod \"cluster-monitoring-operator-64786d9cbb-ckggg\" (UID: \"e13f32f9-100f-456f-bb39-1dac54b64af9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" Jul 05 11:54:26.285578 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.285515 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vw5m\" (UniqueName: \"kubernetes.io/projected/6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568-kube-api-access-8vw5m\") pod \"klusterlet-addon-workmgr-55ff98c4d7-ppr5x\" (UID: \"6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:26.347463 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.347429 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:26.373468 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373438 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-hub\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.373665 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373475 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842db65b-2a1c-4d56-bf76-72b3115a18f0-service-ca-bundle\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.373665 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373500 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/842db65b-2a1c-4d56-bf76-72b3115a18f0-default-certificate\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.373665 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373531 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.373665 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373553 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842db65b-2a1c-4d56-bf76-72b3115a18f0-metrics-certs\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.373665 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373607 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8hxm\" (UniqueName: \"kubernetes.io/projected/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-kube-api-access-c8hxm\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.373665 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373645 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-ca\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.374155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373671 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.374155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373711 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h9nz\" (UniqueName: \"kubernetes.io/projected/c635fb3b-5ece-42e4-a97b-364b7c7f99ee-kube-api-access-5h9nz\") pod \"managed-serviceaccount-addon-agent-b8cd87856-nxzjh\" (UID: \"c635fb3b-5ece-42e4-a97b-364b7c7f99ee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh" Jul 05 11:54:26.374155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373752 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/842db65b-2a1c-4d56-bf76-72b3115a18f0-stats-auth\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.374155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373784 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.374155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373832 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c635fb3b-5ece-42e4-a97b-364b7c7f99ee-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b8cd87856-nxzjh\" (UID: \"c635fb3b-5ece-42e4-a97b-364b7c7f99ee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh" Jul 05 11:54:26.374155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.373896 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhcgb\" (UniqueName: \"kubernetes.io/projected/842db65b-2a1c-4d56-bf76-72b3115a18f0-kube-api-access-zhcgb\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.376207 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.375674 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842db65b-2a1c-4d56-bf76-72b3115a18f0-service-ca-bundle\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.377488 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.376844 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/842db65b-2a1c-4d56-bf76-72b3115a18f0-metrics-certs\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.377488 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.377002 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.377488 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.377444 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-hub\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.378812 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.378787 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.380121 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.380077 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/842db65b-2a1c-4d56-bf76-72b3115a18f0-stats-auth\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.381344 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.381288 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/842db65b-2a1c-4d56-bf76-72b3115a18f0-default-certificate\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.383292 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.383245 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.385522 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.385227 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-ca\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.385700 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.385655 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c635fb3b-5ece-42e4-a97b-364b7c7f99ee-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b8cd87856-nxzjh\" (UID: \"c635fb3b-5ece-42e4-a97b-364b7c7f99ee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh" Jul 05 11:54:26.385700 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.385664 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhcgb\" (UniqueName: \"kubernetes.io/projected/842db65b-2a1c-4d56-bf76-72b3115a18f0-kube-api-access-zhcgb\") pod \"router-default-77bbc49f9c-hdjwj\" (UID: \"842db65b-2a1c-4d56-bf76-72b3115a18f0\") " pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.386156 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.386135 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8hxm\" (UniqueName: \"kubernetes.io/projected/ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef-kube-api-access-c8hxm\") pod \"cluster-proxy-proxy-agent-7bbbc4b577-xvv2k\" (UID: \"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.387033 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.387009 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h9nz\" (UniqueName: \"kubernetes.io/projected/c635fb3b-5ece-42e4-a97b-364b7c7f99ee-kube-api-access-5h9nz\") pod \"managed-serviceaccount-addon-agent-b8cd87856-nxzjh\" (UID: \"c635fb3b-5ece-42e4-a97b-364b7c7f99ee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh" Jul 05 11:54:26.398742 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.398715 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm" Jul 05 11:54:26.413086 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.413061 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" Jul 05 11:54:26.445735 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.445704 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:26.450579 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.450558 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:26.489201 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.489164 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" Jul 05 11:54:26.499900 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.499859 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:26.523368 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:26.523172 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh" Jul 05 11:54:27.276280 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:27.276217 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:27.327018 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:27.326977 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:28.364823 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.364654 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dhzh9"] Jul 05 11:54:28.396102 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.395739 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77bbc49f9c-hdjwj"] Jul 05 11:54:28.405121 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.405082 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k"] Jul 05 11:54:28.439831 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.439773 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg"] Jul 05 11:54:28.550236 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:54:28.550200 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793d5b93_3b3a_4f7b_99fb_38db6730bfa6.slice/crio-b1073b4b32ef02da1b4987b9509a406466ac928fd357219ec4b45a7bcfe6859c WatchSource:0}: Error finding container b1073b4b32ef02da1b4987b9509a406466ac928fd357219ec4b45a7bcfe6859c: Status 404 returned error can't find the container with id b1073b4b32ef02da1b4987b9509a406466ac928fd357219ec4b45a7bcfe6859c Jul 05 11:54:28.551032 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:54:28.550980 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod842db65b_2a1c_4d56_bf76_72b3115a18f0.slice/crio-43413c65cb427813f89ff17e05e9ec1fef355c01d5caa78e3aa47f14225910b1 WatchSource:0}: Error finding container 43413c65cb427813f89ff17e05e9ec1fef355c01d5caa78e3aa47f14225910b1: Status 404 returned error can't find the container with id 43413c65cb427813f89ff17e05e9ec1fef355c01d5caa78e3aa47f14225910b1 Jul 05 11:54:28.552178 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:54:28.552048 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba4a25f5_1a49_4d48_8fa3_3e3e9f3ab3ef.slice/crio-e1550b3f4ae2c1af57f1e48891267accd40c98453f3b40e7c33922d2b0cee653 WatchSource:0}: Error finding container e1550b3f4ae2c1af57f1e48891267accd40c98453f3b40e7c33922d2b0cee653: Status 404 returned error can't find the container with id e1550b3f4ae2c1af57f1e48891267accd40c98453f3b40e7c33922d2b0cee653 Jul 05 11:54:28.552928 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:54:28.552906 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13f32f9_100f_456f_bb39_1dac54b64af9.slice/crio-cc5fe25762214ed19638f9143e4f6636fb453148e58b206d4cad6d44771e8fd4 WatchSource:0}: Error finding container cc5fe25762214ed19638f9143e4f6636fb453148e58b206d4cad6d44771e8fd4: Status 404 returned error can't find the container with id cc5fe25762214ed19638f9143e4f6636fb453148e58b206d4cad6d44771e8fd4 Jul 05 11:54:28.635628 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.635411 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 11:54:28.635628 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.635534 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" event={"ID":"5f7f19ec-9537-4581-b1ad-62d17debda03","Type":"ContainerStarted","Data":"b7c0d00215253b35b99f54c1c2d95677a9cd32e2297c9a596f2d3d4371ec7d34"} Jul 05 11:54:28.643096 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.643041 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:54:28.643795 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.643724 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gplkw" event={"ID":"31952da1-f50a-4817-b342-775273f214f5","Type":"ContainerStarted","Data":"1b525c692d6807c16b4af6662f266a5a6ea77840a27bd3c9004919cf5f9875f2"} Jul 05 11:54:28.646208 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.646184 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" event={"ID":"842db65b-2a1c-4d56-bf76-72b3115a18f0","Type":"ContainerStarted","Data":"43413c65cb427813f89ff17e05e9ec1fef355c01d5caa78e3aa47f14225910b1"} Jul 05 11:54:28.647977 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.647626 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dhzh9" event={"ID":"793d5b93-3b3a-4f7b-99fb-38db6730bfa6","Type":"ContainerStarted","Data":"b1073b4b32ef02da1b4987b9509a406466ac928fd357219ec4b45a7bcfe6859c"} Jul 05 11:54:28.650088 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.650063 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" event={"ID":"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef","Type":"ContainerStarted","Data":"e1550b3f4ae2c1af57f1e48891267accd40c98453f3b40e7c33922d2b0cee653"} Jul 05 11:54:28.652767 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.652743 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" event={"ID":"8efd845e-7e3b-4a9f-9572-43d72a6a22f0","Type":"ContainerStarted","Data":"19efea5fe238fe5babe3b407c7a3b0944d119a581d6c1fa3a9b372a77ebd437e"} Jul 05 11:54:28.654213 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.654179 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" event={"ID":"e13f32f9-100f-456f-bb39-1dac54b64af9","Type":"ContainerStarted","Data":"cc5fe25762214ed19638f9143e4f6636fb453148e58b206d4cad6d44771e8fd4"} Jul 05 11:54:28.662378 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.662350 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm"] Jul 05 11:54:28.663958 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.663854 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh"] Jul 05 11:54:28.664841 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.664779 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" podStartSLOduration=50.171367008 podStartE2EDuration="1m0.664759386s" totalImagesPullingTime="10.493392378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:28.660709558 +0000 UTC m=+69.051411321" watchObservedRunningTime="2026-07-05 11:54:28.664759386 +0000 UTC m=+69.055461150" Jul 05 11:54:28.665135 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:54:28.665115 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc843d931_14bd_431f_9263_9671ea3401a8.slice/crio-7bb0e8d0e630e21ddd941ae2659a565a2c8fc8ca13b8e87b55cece8d1c024158 WatchSource:0}: Error finding container 7bb0e8d0e630e21ddd941ae2659a565a2c8fc8ca13b8e87b55cece8d1c024158: Status 404 returned error can't find the container with id 7bb0e8d0e630e21ddd941ae2659a565a2c8fc8ca13b8e87b55cece8d1c024158 Jul 05 11:54:28.668886 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:54:28.668644 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f6a91d4_f6ba_4113_9e05_0a2f4d8dc568.slice/crio-42f9bd1cd76ca11cf9e6851f9687249ddc6c33d9fc6af82189ea7c727359e6fc WatchSource:0}: Error finding container 42f9bd1cd76ca11cf9e6851f9687249ddc6c33d9fc6af82189ea7c727359e6fc: Status 404 returned error can't find the container with id 42f9bd1cd76ca11cf9e6851f9687249ddc6c33d9fc6af82189ea7c727359e6fc Jul 05 11:54:28.670154 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.670037 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x"] Jul 05 11:54:28.894524 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:28.894448 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-4sth7" Jul 05 11:54:29.080422 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.080387 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:29.138946 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.138765 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:29.234349 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.234251 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:29.377304 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.377221 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:29.549216 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.549019 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:29.670265 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.670226 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" event={"ID":"8efd845e-7e3b-4a9f-9572-43d72a6a22f0","Type":"ContainerStarted","Data":"4fd8afd224f651b2c91e5c3e557fa0bd399762b6c66fd7199e27f02be65ea0a4"} Jul 05 11:54:29.689357 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.688470 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-479rr" podStartSLOduration=56.453131894 podStartE2EDuration="1m1.688449609s" totalImagesPullingTime="5.235317715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:29.686222664 +0000 UTC m=+70.076924428" watchObservedRunningTime="2026-07-05 11:54:29.688449609 +0000 UTC m=+70.079151353" Jul 05 11:54:29.692391 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.692283 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" event={"ID":"6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568","Type":"ContainerStarted","Data":"42f9bd1cd76ca11cf9e6851f9687249ddc6c33d9fc6af82189ea7c727359e6fc"} Jul 05 11:54:29.696903 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.696826 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh" event={"ID":"c635fb3b-5ece-42e4-a97b-364b7c7f99ee","Type":"ContainerStarted","Data":"e0b51a5525057e9e1562945a3090b458a846e4b434962c4a3d121b1f1f7159a4"} Jul 05 11:54:29.703518 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.702707 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gplkw" event={"ID":"31952da1-f50a-4817-b342-775273f214f5","Type":"ContainerStarted","Data":"8e629b74b17d2130ba053d703072f0f7f11e63ab889fa780b409b4e1bb69851e"} Jul 05 11:54:29.703518 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.703312 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gplkw" Jul 05 11:54:29.716588 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.716548 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" event={"ID":"842db65b-2a1c-4d56-bf76-72b3115a18f0","Type":"ContainerStarted","Data":"96ab5f3db6452cfb9f7fe4673ce0f86c14217ffc453b0b9265ed4d07fcec6ef7"} Jul 05 11:54:29.726028 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.725699 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gplkw" podStartSLOduration=34.533244928 podStartE2EDuration="39.72567823s" totalImagesPullingTime="5.192433302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:29.723944096 +0000 UTC m=+70.114645859" watchObservedRunningTime="2026-07-05 11:54:29.72567823 +0000 UTC m=+70.116379993" Jul 05 11:54:29.731931 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.731904 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m7tqb" event={"ID":"42dcc512-1104-4580-a06a-42db9e8317f7","Type":"ContainerStarted","Data":"c2f2c7a43f1b97d36510a217b27105d2274df7169d38334b87ee11283caa4674"} Jul 05 11:54:29.737752 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.737725 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm" event={"ID":"c843d931-14bd-431f-9263-9671ea3401a8","Type":"ContainerStarted","Data":"7bb0e8d0e630e21ddd941ae2659a565a2c8fc8ca13b8e87b55cece8d1c024158"} Jul 05 11:54:29.747676 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.745961 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" podStartSLOduration=3.745942638 podStartE2EDuration="3.745942638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:29.744954944 +0000 UTC m=+70.135656707" watchObservedRunningTime="2026-07-05 11:54:29.745942638 +0000 UTC m=+70.136644398" Jul 05 11:54:29.749290 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:29.749198 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:30.141595 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:30.141563 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:30.247840 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:30.247786 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:30.500513 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:30.500423 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:30.504202 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:30.504039 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:30.744375 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:30.744324 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:30.746153 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:30.745966 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-77bbc49f9c-hdjwj" Jul 05 11:54:30.952847 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:30.952725 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:31.154112 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:31.154024 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:31.202913 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:31.202797 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:31.778005 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:31.777934 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:31.876957 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:31.876913 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:34.274288 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:34.274188 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:34.328088 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:34.328036 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:35.530579 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:35.530543 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rt4k7" Jul 05 11:54:36.625902 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.625853 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:36.724213 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.724175 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:36.771945 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.771900 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" event={"ID":"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef","Type":"ContainerStarted","Data":"2c3edbf938049fe83673b24d6aa090e9d9e08596e1d7181f5a1a83216cb7e69e"} Jul 05 11:54:36.772136 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.771953 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" event={"ID":"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef","Type":"ContainerStarted","Data":"6eb9f07c1bc5b2bcf1dc7a9a0e2a65e9274e6fd74a9fc4d3f92ffa78280cb457"} Jul 05 11:54:36.772136 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.771974 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" event={"ID":"ba4a25f5-1a49-4d48-8fa3-3e3e9f3ab3ef","Type":"ContainerStarted","Data":"c18fbe04bdcaabdbfa3ae45ea088011657e2bd11cb0b234fdd2141e2f833ba1d"} Jul 05 11:54:36.773780 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.773497 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" event={"ID":"e13f32f9-100f-456f-bb39-1dac54b64af9","Type":"ContainerStarted","Data":"0d1c60f713b7a5032b4e10f6a0d2d37f1d50a73af2b83e9cdf777e875e12951a"} Jul 05 11:54:36.775102 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.775074 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" event={"ID":"6f6a91d4-f6ba-4113-9e05-0a2f4d8dc568","Type":"ContainerStarted","Data":"e535a23c7ec2f4a711c6ad8cdf407672611d6cd4d9520d547efd029db5679073"} Jul 05 11:54:36.775245 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.775143 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:36.776765 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.776702 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh" event={"ID":"c635fb3b-5ece-42e4-a97b-364b7c7f99ee","Type":"ContainerStarted","Data":"81edf4f6aaa0a779eb03c5dbf47484fc505e7071dd04a036fb020bae07bdbd2c"} Jul 05 11:54:36.777022 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.776997 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" Jul 05 11:54:36.780794 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.780766 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m7tqb" event={"ID":"42dcc512-1104-4580-a06a-42db9e8317f7","Type":"ContainerStarted","Data":"bda3ec9e12abb018f35a1e6cd579dc4134935d69e0b227a88aa327d60e6f8504"} Jul 05 11:54:36.783286 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.783262 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dhzh9" event={"ID":"793d5b93-3b3a-4f7b-99fb-38db6730bfa6","Type":"ContainerStarted","Data":"8055b8cd7bad5269047c93bc9dee489af05d8e48ef5f70749c2580c11f2a02dc"} Jul 05 11:54:36.783399 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.783309 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dhzh9" event={"ID":"793d5b93-3b3a-4f7b-99fb-38db6730bfa6","Type":"ContainerStarted","Data":"7f49751fa6457b8fec452c1de425c3ff7ff58fa501b61cf492462d4b832bb0ef"} Jul 05 11:54:36.784681 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.784660 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm" event={"ID":"c843d931-14bd-431f-9263-9671ea3401a8","Type":"ContainerStarted","Data":"9bb3b8ab2a9b270d574128d23af2ab773f33d56a580c41bf0c1da8db36017c35"} Jul 05 11:54:36.792169 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.792144 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:36.792365 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.792229 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bbbc4b577-xvv2k" podStartSLOduration=3.2536725779999998 podStartE2EDuration="10.792211799s" totalImagesPullingTime="7.538539221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:36.790671913 +0000 UTC m=+77.181373679" watchObservedRunningTime="2026-07-05 11:54:36.792211799 +0000 UTC m=+77.182913563" Jul 05 11:54:36.812979 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.812481 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dhzh9" podStartSLOduration=70.436435472 podStartE2EDuration="1m15.812467446s" totalImagesPullingTime="5.376031974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:36.811160678 +0000 UTC m=+77.201862440" watchObservedRunningTime="2026-07-05 11:54:36.812467446 +0000 UTC m=+77.203169202" Jul 05 11:54:36.846045 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.845708 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ckggg" podStartSLOduration=3.306135039 podStartE2EDuration="10.845689476s" totalImagesPullingTime="7.539554437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:36.844545252 +0000 UTC m=+77.235247016" watchObservedRunningTime="2026-07-05 11:54:36.845689476 +0000 UTC m=+77.236391239" Jul 05 11:54:36.846302 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.846268 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-8pfwm" podStartSLOduration=3.406271773 podStartE2EDuration="10.846257724s" totalImagesPullingTime="7.439985951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:36.827138063 +0000 UTC m=+77.217839826" watchObservedRunningTime="2026-07-05 11:54:36.846257724 +0000 UTC m=+77.236959468" Jul 05 11:54:36.877000 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.876948 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55ff98c4d7-ppr5x" podStartSLOduration=3.416693798 podStartE2EDuration="10.876926587s" totalImagesPullingTime="7.460232789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:36.872779767 +0000 UTC m=+77.263481534" watchObservedRunningTime="2026-07-05 11:54:36.876926587 +0000 UTC m=+77.267628350" Jul 05 11:54:36.902230 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.902173 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b8cd87856-nxzjh" podStartSLOduration=3.46390198 podStartE2EDuration="10.902155823s" totalImagesPullingTime="7.438253843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:36.901776591 +0000 UTC m=+77.292478353" watchObservedRunningTime="2026-07-05 11:54:36.902155823 +0000 UTC m=+77.292857627" Jul 05 11:54:36.939086 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:36.939049 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:37.039402 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:37.039365 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:37.145681 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:37.145588 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:37.195322 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:37.195287 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:37.736338 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:37.736293 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:37.874060 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:37.873995 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:37.973781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:37.973735 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:38.269967 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:38.269917 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:38.317779 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:38.317740 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:39.800640 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:39.800600 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m7tqb" event={"ID":"42dcc512-1104-4580-a06a-42db9e8317f7","Type":"ContainerStarted","Data":"e4786cb844b57e25ce72118f4eba545f93b2e27348c845c8861caa6a4767b8b1"} Jul 05 11:54:39.818129 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:39.818085 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m7tqb" podStartSLOduration=18.340778102 podStartE2EDuration="34.818069717s" totalImagesPullingTime="16.477291615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:54:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:39.815824123 +0000 UTC m=+80.206525887" watchObservedRunningTime="2026-07-05 11:54:39.818069717 +0000 UTC m=+80.208771480" Jul 05 11:54:40.750073 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:40.750044 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gplkw" Jul 05 11:54:41.247150 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:41.247109 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:41.347632 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:41.347595 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:41.448323 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:41.448292 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:41.801160 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:41.801123 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:41.869592 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:41.869558 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:42.034070 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:42.034031 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:42.083384 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:42.083351 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:42.133898 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:42.133854 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:42.219341 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:42.219225 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:42.564750 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:42.564716 2575 patch_prober.go:28] interesting pod/image-registry-96db66bd7-drwx2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 05 11:54:42.565144 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:42.564771 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" podUID="05dde2fd-fba0-412f-90a3-97cc45e9c537" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 05 11:54:43.982255 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:43.982222 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:44.032930 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:44.032869 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:44.614861 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:44.614829 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:54:44.710234 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:44.710192 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:44.877708 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:44.877609 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:44.927090 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:44.927060 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:46.936630 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:46.936581 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:47.032940 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.032899 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:47.141288 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.141255 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:47.202520 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.202433 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7d6dg"] Jul 05 11:54:47.207568 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.207546 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.210452 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.210429 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 05 11:54:47.210577 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.210429 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 05 11:54:47.210577 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.210566 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 05 11:54:47.211690 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.211669 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mqzr7\"" Jul 05 11:54:47.211790 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.211685 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 05 11:54:47.228377 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.228347 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:47.366502 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.366453 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:47.371408 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.371383 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-wtmp\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.371508 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.371427 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-textfile\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.371508 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.371446 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-tls\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.371631 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.371527 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-root\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.371631 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.371587 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-sys\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.371631 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.371616 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.371774 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.371648 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-metrics-client-ca\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.371774 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.371692 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-accelerators-collector-config\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.371774 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.371740 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv8d6\" (UniqueName: \"kubernetes.io/projected/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-kube-api-access-hv8d6\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.464278 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.464185 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:47.473509 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473477 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-textfile\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.473680 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473514 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-tls\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.473680 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473550 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-root\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.473680 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473603 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-sys\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.473680 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473629 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.473680 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473660 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-metrics-client-ca\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.473680 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473663 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-root\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.474008 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473731 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-sys\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.474008 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473795 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-accelerators-collector-config\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.474008 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473850 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-textfile\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.474008 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473863 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv8d6\" (UniqueName: \"kubernetes.io/projected/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-kube-api-access-hv8d6\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.474008 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.473926 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-wtmp\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.474248 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.474089 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-wtmp\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.474436 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.474409 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-metrics-client-ca\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.474819 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.474789 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-accelerators-collector-config\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.476082 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.476058 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.476212 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.476143 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-node-exporter-tls\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.481863 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.481843 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv8d6\" (UniqueName: \"kubernetes.io/projected/57363bc0-e79d-4f28-b2e9-35a3a9c63b1a-kube-api-access-hv8d6\") pod \"node-exporter-7d6dg\" (UID: \"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a\") " pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.519394 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.519354 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7d6dg" Jul 05 11:54:47.532033 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:54:47.531995 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57363bc0_e79d_4f28_b2e9_35a3a9c63b1a.slice/crio-74dcabd8c3f188e3955943562c2e979be8ded6a24197b9d3ffcd57f3716f511c WatchSource:0}: Error finding container 74dcabd8c3f188e3955943562c2e979be8ded6a24197b9d3ffcd57f3716f511c: Status 404 returned error can't find the container with id 74dcabd8c3f188e3955943562c2e979be8ded6a24197b9d3ffcd57f3716f511c Jul 05 11:54:47.564715 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.564674 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:47.636303 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.636260 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:47.740239 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.740138 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:47.834307 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.834263 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7d6dg" event={"ID":"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a","Type":"ContainerStarted","Data":"74dcabd8c3f188e3955943562c2e979be8ded6a24197b9d3ffcd57f3716f511c"} Jul 05 11:54:47.841776 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:47.841748 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:48.152031 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:48.151994 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:48.229257 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:48.229223 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:48.692639 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:48.692612 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:48.839186 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:48.839148 2575 generic.go:356] "Generic (PLEG): container finished" podID="57363bc0-e79d-4f28-b2e9-35a3a9c63b1a" containerID="6e986840e82c1ae8c0144edc9785763a09fe0effa989912669bbcb8046fbb6cc" exitCode=0 Jul 05 11:54:48.839348 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:48.839211 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7d6dg" event={"ID":"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a","Type":"ContainerDied","Data":"6e986840e82c1ae8c0144edc9785763a09fe0effa989912669bbcb8046fbb6cc"} Jul 05 11:54:49.129071 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:49.129038 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:49.232351 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:49.232260 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:49.287309 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:49.287276 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:49.666625 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:49.666585 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-96db66bd7-drwx2"] Jul 05 11:54:49.742607 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:49.742575 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:49.841573 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:49.841532 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:49.845475 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:49.845445 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7d6dg" event={"ID":"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a","Type":"ContainerStarted","Data":"2393555f29b17f925fd3ac2e62158bafc8f8c04ae1929b3c11b09a4b80400cf9"} Jul 05 11:54:49.845632 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:49.845499 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7d6dg" event={"ID":"57363bc0-e79d-4f28-b2e9-35a3a9c63b1a","Type":"ContainerStarted","Data":"b76d2f0345025c85cf0f72577268f1a641b18b1a345da82536bab4f4bd99c908"} Jul 05 11:54:49.863451 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:49.863395 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7d6dg" podStartSLOduration=1.711572353 podStartE2EDuration="2.86338202s" totalImagesPullingTime="1.086838803s" totalInitContainerRuntime="64.970864ms" isStatefulPod=true podCreationTimestamp="2026-07-05 11:54:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:49.861889029 +0000 UTC m=+90.252590792" watchObservedRunningTime="2026-07-05 11:54:49.86338202 +0000 UTC m=+90.254083787" Jul 05 11:54:50.078703 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:50.078661 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:50.157162 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:50.157097 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:50.246048 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:50.246006 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:50.338397 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:50.338296 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:51.689155 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:51.689115 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:51.737660 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:51.737618 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:51.963304 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:51.963218 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:52.014781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:52.014750 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:52.548573 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:52.548533 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:53.536516 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:53.536478 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:55.630392 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:55.630355 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:55.730171 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:55.730133 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:56.859949 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:56.859902 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:56.960218 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:56.960180 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:57.188532 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:57.188422 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:57.285113 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:57.285075 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:58.880292 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:58.880253 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:54:59.990458 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:54:59.990400 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:00.091679 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:00.091634 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:00.191355 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:00.191308 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:00.761022 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:00.760976 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:00.865330 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:00.865296 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:04.288103 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:04.288065 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:04.386340 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:04.386294 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:04.487138 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:04.487110 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:13.326480 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:13.326443 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:14.686961 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:14.686919 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" podUID="05dde2fd-fba0-412f-90a3-97cc45e9c537" containerName="registry" containerID="cri-o://abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b" gracePeriod=30 Jul 05 11:55:14.929995 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:14.929968 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:55:14.945347 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:14.945265 2575 generic.go:356] "Generic (PLEG): container finished" podID="05dde2fd-fba0-412f-90a3-97cc45e9c537" containerID="abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b" exitCode=0 Jul 05 11:55:14.945347 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:14.945307 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" Jul 05 11:55:14.945502 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:14.945355 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" event={"ID":"05dde2fd-fba0-412f-90a3-97cc45e9c537","Type":"ContainerDied","Data":"abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b"} Jul 05 11:55:14.945502 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:14.945404 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96db66bd7-drwx2" event={"ID":"05dde2fd-fba0-412f-90a3-97cc45e9c537","Type":"ContainerDied","Data":"cbed1868fa2c316f25763be5c939f86119d7210fa53598c37dd4bfe2b15a1ac5"} Jul 05 11:55:14.945502 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:14.945425 2575 scope.go:117] "RemoveContainer" containerID="abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b" Jul 05 11:55:14.954644 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:14.954626 2575 scope.go:117] "RemoveContainer" containerID="abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b" Jul 05 11:55:14.955002 ip-10-0-142-207 kubenswrapper[2575]: E0705 11:55:14.954966 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b\": container with ID starting with abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b not found: ID does not exist" containerID="abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b" Jul 05 11:55:14.955124 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:14.955011 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b"} err="failed to get container status \"abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b\": rpc error: code = NotFound desc = could not find container \"abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b\": container with ID starting with abf1393bcb1a5c947367503f24d22baec454dfe8062151247a0d483eccfeaf3b not found: ID does not exist" Jul 05 11:55:15.008246 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.008207 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-bound-sa-token\") pod \"05dde2fd-fba0-412f-90a3-97cc45e9c537\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " Jul 05 11:55:15.008246 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.008252 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-installation-pull-secrets\") pod \"05dde2fd-fba0-412f-90a3-97cc45e9c537\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " Jul 05 11:55:15.008481 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.008287 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-certificates\") pod \"05dde2fd-fba0-412f-90a3-97cc45e9c537\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " Jul 05 11:55:15.008481 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.008309 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2q2tp\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-kube-api-access-2q2tp\") pod \"05dde2fd-fba0-412f-90a3-97cc45e9c537\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " Jul 05 11:55:15.008481 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.008337 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dde2fd-fba0-412f-90a3-97cc45e9c537-ca-trust-extracted\") pod \"05dde2fd-fba0-412f-90a3-97cc45e9c537\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " Jul 05 11:55:15.008481 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.008378 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-trusted-ca\") pod \"05dde2fd-fba0-412f-90a3-97cc45e9c537\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " Jul 05 11:55:15.008685 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.008617 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-image-registry-private-configuration\") pod \"05dde2fd-fba0-412f-90a3-97cc45e9c537\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " Jul 05 11:55:15.008685 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.008681 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls\") pod \"05dde2fd-fba0-412f-90a3-97cc45e9c537\" (UID: \"05dde2fd-fba0-412f-90a3-97cc45e9c537\") " Jul 05 11:55:15.008807 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.008765 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "05dde2fd-fba0-412f-90a3-97cc45e9c537" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:55:15.009058 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.008866 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "05dde2fd-fba0-412f-90a3-97cc45e9c537" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:55:15.009058 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.009016 2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-certificates\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 11:55:15.009058 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.009034 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dde2fd-fba0-412f-90a3-97cc45e9c537-trusted-ca\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 11:55:15.010852 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.010825 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "05dde2fd-fba0-412f-90a3-97cc45e9c537" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 11:55:15.010977 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.010832 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "05dde2fd-fba0-412f-90a3-97cc45e9c537" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 11:55:15.010977 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.010911 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "05dde2fd-fba0-412f-90a3-97cc45e9c537" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:55:15.011068 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.011013 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-kube-api-access-2q2tp" (OuterVolumeSpecName: "kube-api-access-2q2tp") pod "05dde2fd-fba0-412f-90a3-97cc45e9c537" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537"). InnerVolumeSpecName "kube-api-access-2q2tp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 11:55:15.011113 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.011089 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "05dde2fd-fba0-412f-90a3-97cc45e9c537" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:55:15.017190 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.017164 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05dde2fd-fba0-412f-90a3-97cc45e9c537-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "05dde2fd-fba0-412f-90a3-97cc45e9c537" (UID: "05dde2fd-fba0-412f-90a3-97cc45e9c537"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 11:55:15.109818 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.109786 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-registry-tls\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 11:55:15.109818 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.109814 2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-bound-sa-token\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 11:55:15.110037 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.109827 2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-installation-pull-secrets\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 11:55:15.110037 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.109841 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q2tp\" (UniqueName: \"kubernetes.io/projected/05dde2fd-fba0-412f-90a3-97cc45e9c537-kube-api-access-2q2tp\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 11:55:15.110037 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.109853 2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dde2fd-fba0-412f-90a3-97cc45e9c537-ca-trust-extracted\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 11:55:15.110037 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.109866 2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dde2fd-fba0-412f-90a3-97cc45e9c537-image-registry-private-configuration\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 11:55:15.266863 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.266830 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-96db66bd7-drwx2"] Jul 05 11:55:15.270260 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:15.270234 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-96db66bd7-drwx2"] Jul 05 11:55:16.128624 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:16.128587 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05dde2fd-fba0-412f-90a3-97cc45e9c537" path="/var/lib/kubelet/pods/05dde2fd-fba0-412f-90a3-97cc45e9c537/volumes" Jul 05 11:55:19.275448 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:19.275407 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:19.322209 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:19.322125 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:19.968439 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:19.968402 2575 generic.go:356] "Generic (PLEG): container finished" podID="1881b431-0b27-4e0f-a16f-6a0e93cafaa3" containerID="fb2f1b9afc6449cd519c7b2caf45b3ffe52f2fa79b8a20417057d7e62ee346dc" exitCode=0 Jul 05 11:55:19.968685 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:19.968474 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" event={"ID":"1881b431-0b27-4e0f-a16f-6a0e93cafaa3","Type":"ContainerDied","Data":"fb2f1b9afc6449cd519c7b2caf45b3ffe52f2fa79b8a20417057d7e62ee346dc"} Jul 05 11:55:19.968789 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:19.968776 2575 scope.go:117] "RemoveContainer" containerID="fb2f1b9afc6449cd519c7b2caf45b3ffe52f2fa79b8a20417057d7e62ee346dc" Jul 05 11:55:20.974241 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:20.974208 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-md6wx" event={"ID":"1881b431-0b27-4e0f-a16f-6a0e93cafaa3","Type":"ContainerStarted","Data":"b09be412a3eea0b1c40b386edb51d2ca0be2b59ce1996c270e6fe3151531da5e"} Jul 05 11:55:24.992098 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:24.992001 2575 generic.go:356] "Generic (PLEG): container finished" podID="7197549e-9314-42dd-bbdb-d53feb6ece0e" containerID="7b0f78937de7d9bbbf6630722f2b67779fc4b0abd099289d3f94256c56d15d5a" exitCode=0 Jul 05 11:55:24.992098 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:24.992078 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" event={"ID":"7197549e-9314-42dd-bbdb-d53feb6ece0e","Type":"ContainerDied","Data":"7b0f78937de7d9bbbf6630722f2b67779fc4b0abd099289d3f94256c56d15d5a"} Jul 05 11:55:24.992489 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:24.992397 2575 scope.go:117] "RemoveContainer" containerID="7b0f78937de7d9bbbf6630722f2b67779fc4b0abd099289d3f94256c56d15d5a" Jul 05 11:55:25.997366 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:25.997327 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-kf6hq" event={"ID":"7197549e-9314-42dd-bbdb-d53feb6ece0e","Type":"ContainerStarted","Data":"1a76ecbb8d91b42ee4ca376c67fa435dc58a050dc9bc29d2284dbf00fea9b215"} Jul 05 11:55:29.122033 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:29.121991 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:29.172684 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:29.172646 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:55:33.025857 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:33.025821 2575 generic.go:356] "Generic (PLEG): container finished" podID="be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34" containerID="3c721e69a435f947f3adb911eabdf396c1b5b8770cc7599dbb7dd95795eebd84" exitCode=0 Jul 05 11:55:33.026270 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:33.025894 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" event={"ID":"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34","Type":"ContainerDied","Data":"3c721e69a435f947f3adb911eabdf396c1b5b8770cc7599dbb7dd95795eebd84"} Jul 05 11:55:33.026270 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:33.026204 2575 scope.go:117] "RemoveContainer" containerID="3c721e69a435f947f3adb911eabdf396c1b5b8770cc7599dbb7dd95795eebd84" Jul 05 11:55:34.032970 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:55:34.032332 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-j8f2q" event={"ID":"be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34","Type":"ContainerStarted","Data":"e85599e83ab04dffc8b271563b655965e779ceac3ff730a3e7b42580aacd741b"} Jul 05 11:56:06.353371 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:56:06.353332 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:56:06.554027 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:56:06.553988 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:56:15.261566 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:56:15.261490 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:56:15.405518 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:56:15.405476 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:56:15.506366 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:56:15.506323 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:56:16.009417 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:56:16.009383 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:56:26.643201 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:56:26.643157 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:56:26.744021 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:56:26.743987 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:56:26.791823 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:56:26.791789 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:56:55.879605 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:56:55.879570 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:56:55.929470 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:56:55.929442 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:21.562428 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:21.562391 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:21.614593 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:21.614564 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:29.220555 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:29.220518 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:29.268213 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:29.268171 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:29.367480 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:29.367441 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:29.468679 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:29.468617 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:30.187386 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:30.187348 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:30.235431 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:30.235397 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:33.557761 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:33.557722 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:33.603710 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:33.603676 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:39.912851 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:39.912808 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:39.962015 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:39.961980 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:40.014153 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:40.014117 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:49.288738 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:49.288648 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:50.415013 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:50.414973 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:57:58.272577 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:57:58.272529 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:58:20.028999 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:58:20.028974 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 11:58:20.035441 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:58:20.035422 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 11:58:20.039576 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:58:20.039555 2575 kubelet.go:1648] "Image garbage collection succeeded" Jul 05 11:58:36.438068 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:58:36.438029 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:58:36.536503 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:58:36.536470 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:58:58.012523 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:58:58.012485 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:58:58.791314 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:58:58.791277 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:01.613219 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:01.613177 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:01.669050 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:01.669008 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:03.455053 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:03.455012 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:03.565415 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:03.565374 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:05.813002 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:05.812958 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:06.227559 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:06.227467 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:08.415160 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:08.415123 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:08.464945 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:08.464915 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:14.018968 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:14.018868 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:16.088307 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:16.088272 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:16.183558 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:16.183522 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:21.919118 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:21.919078 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:22.019337 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:22.019304 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:24.085597 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:24.085557 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:35.393357 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:35.393320 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:35.440568 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:35.440535 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:41.759011 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:41.758977 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:50.400854 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:50.400803 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:50.515675 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:50.515637 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:50.655036 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:50.654941 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:50.754941 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:50.754894 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:51.268669 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.268628 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:51.527525 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.527435 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:51.629559 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.629522 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:51.654638 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.654609 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh"] Jul 05 11:59:51.654929 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.654915 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05dde2fd-fba0-412f-90a3-97cc45e9c537" containerName="registry" Jul 05 11:59:51.654978 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.654930 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="05dde2fd-fba0-412f-90a3-97cc45e9c537" containerName="registry" Jul 05 11:59:51.655012 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.654989 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="05dde2fd-fba0-412f-90a3-97cc45e9c537" containerName="registry" Jul 05 11:59:51.657779 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.657763 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.660975 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.660953 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 05 11:59:51.661236 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.661212 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 05 11:59:51.661640 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.661619 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ctxds\"" Jul 05 11:59:51.661781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.661760 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 05 11:59:51.661857 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.661690 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 05 11:59:51.670191 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.670166 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh"] Jul 05 11:59:51.773781 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.773748 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:51.790801 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.790723 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a1f1d34-d351-4872-a61d-861362025eac-manifests\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.790801 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.790771 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lthrb\" (UniqueName: \"kubernetes.io/projected/8a1f1d34-d351-4872-a61d-861362025eac-kube-api-access-lthrb\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.790801 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.790791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a1f1d34-d351-4872-a61d-861362025eac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.791074 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.790925 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a1f1d34-d351-4872-a61d-861362025eac-tmp\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.791074 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.790963 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a1f1d34-d351-4872-a61d-861362025eac-webhook-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.872820 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.872783 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:51.891657 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.891626 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lthrb\" (UniqueName: \"kubernetes.io/projected/8a1f1d34-d351-4872-a61d-861362025eac-kube-api-access-lthrb\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.891808 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.891663 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a1f1d34-d351-4872-a61d-861362025eac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.891808 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.891700 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a1f1d34-d351-4872-a61d-861362025eac-tmp\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.891808 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.891746 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a1f1d34-d351-4872-a61d-861362025eac-webhook-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.891808 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.891804 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a1f1d34-d351-4872-a61d-861362025eac-manifests\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.892270 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.892241 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a1f1d34-d351-4872-a61d-861362025eac-manifests\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.894074 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.894054 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a1f1d34-d351-4872-a61d-861362025eac-tmp\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.894181 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.894163 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a1f1d34-d351-4872-a61d-861362025eac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.894303 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.894284 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a1f1d34-d351-4872-a61d-861362025eac-webhook-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.901287 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.901263 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lthrb\" (UniqueName: \"kubernetes.io/projected/8a1f1d34-d351-4872-a61d-861362025eac-kube-api-access-lthrb\") pod \"opendatahub-operator-controller-manager-59595c8d98-96cwh\" (UID: \"8a1f1d34-d351-4872-a61d-861362025eac\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.968415 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.968382 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:51.975654 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:51.975627 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:52.075036 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:52.074994 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:52.104152 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:52.104121 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh"] Jul 05 11:59:52.107419 ip-10-0-142-207 kubenswrapper[2575]: W0705 11:59:52.107391 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1f1d34_d351_4872_a61d_861362025eac.slice/crio-43eb36e0b768baae846f9a619485d1b8aaa00d6eae871c9bdb3828950bd85f68 WatchSource:0}: Error finding container 43eb36e0b768baae846f9a619485d1b8aaa00d6eae871c9bdb3828950bd85f68: Status 404 returned error can't find the container with id 43eb36e0b768baae846f9a619485d1b8aaa00d6eae871c9bdb3828950bd85f68 Jul 05 11:59:52.109251 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:52.109231 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 11:59:52.165474 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:52.165442 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:52.226062 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:52.226014 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 11:59:53.020292 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:53.020246 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" event={"ID":"8a1f1d34-d351-4872-a61d-861362025eac","Type":"ContainerStarted","Data":"43eb36e0b768baae846f9a619485d1b8aaa00d6eae871c9bdb3828950bd85f68"} Jul 05 11:59:56.035320 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:56.035280 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" event={"ID":"8a1f1d34-d351-4872-a61d-861362025eac","Type":"ContainerStarted","Data":"741f1ed4771f09abb9c0f8915d765c26bb707c6604808940fbfccb77eeb0be8d"} Jul 05 11:59:57.041338 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:57.041298 2575 generic.go:356] "Generic (PLEG): container finished" podID="8a1f1d34-d351-4872-a61d-861362025eac" containerID="741f1ed4771f09abb9c0f8915d765c26bb707c6604808940fbfccb77eeb0be8d" exitCode=0 Jul 05 11:59:57.041824 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:57.041380 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" event={"ID":"8a1f1d34-d351-4872-a61d-861362025eac","Type":"ContainerDied","Data":"741f1ed4771f09abb9c0f8915d765c26bb707c6604808940fbfccb77eeb0be8d"} Jul 05 11:59:58.047196 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:58.047161 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" event={"ID":"8a1f1d34-d351-4872-a61d-861362025eac","Type":"ContainerStarted","Data":"f2116f729755ba01acdde3743b98957fd87c2af9803a8f4852c8bb8a4e56c6ec"} Jul 05 11:59:58.047666 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:58.047275 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 11:59:58.067718 ip-10-0-142-207 kubenswrapper[2575]: I0705 11:59:58.067662 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" podStartSLOduration=2.785316802 podStartE2EDuration="7.067647022s" totalImagesPullingTime="3.269611016s" totalInitContainerRuntime="1.012719204s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:59:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:59:58.066108728 +0000 UTC m=+398.456810490" watchObservedRunningTime="2026-07-05 11:59:58.067647022 +0000 UTC m=+398.458348783" Jul 05 12:00:09.053681 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:09.053650 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-96cwh" Jul 05 12:00:11.490319 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:11.490282 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:11.597173 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:11.597136 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:11.846615 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:11.846577 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:11.990093 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:11.990057 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658"] Jul 05 12:00:12.003747 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.003719 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" Jul 05 12:00:12.006740 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.006713 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-57829\"" Jul 05 12:00:12.006923 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.006729 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 05 12:00:12.006923 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.006778 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 05 12:00:12.013706 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.013683 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658"] Jul 05 12:00:12.132014 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.131925 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:12.172528 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.172490 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30705649-53ef-425c-940a-f0c8d125e5d4-tmp\") pod \"kube-auth-proxy-5f99dd8ff-jf658\" (UID: \"30705649-53ef-425c-940a-f0c8d125e5d4\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" Jul 05 12:00:12.172685 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.172539 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52lzz\" (UniqueName: \"kubernetes.io/projected/30705649-53ef-425c-940a-f0c8d125e5d4-kube-api-access-52lzz\") pod \"kube-auth-proxy-5f99dd8ff-jf658\" (UID: \"30705649-53ef-425c-940a-f0c8d125e5d4\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" Jul 05 12:00:12.172685 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.172560 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30705649-53ef-425c-940a-f0c8d125e5d4-tls-certs\") pod \"kube-auth-proxy-5f99dd8ff-jf658\" (UID: \"30705649-53ef-425c-940a-f0c8d125e5d4\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" Jul 05 12:00:12.205459 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.205425 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:12.270247 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.270208 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:12.273326 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.273302 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30705649-53ef-425c-940a-f0c8d125e5d4-tmp\") pod \"kube-auth-proxy-5f99dd8ff-jf658\" (UID: \"30705649-53ef-425c-940a-f0c8d125e5d4\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" Jul 05 12:00:12.273477 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.273339 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52lzz\" (UniqueName: \"kubernetes.io/projected/30705649-53ef-425c-940a-f0c8d125e5d4-kube-api-access-52lzz\") pod \"kube-auth-proxy-5f99dd8ff-jf658\" (UID: \"30705649-53ef-425c-940a-f0c8d125e5d4\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" Jul 05 12:00:12.273477 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.273361 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30705649-53ef-425c-940a-f0c8d125e5d4-tls-certs\") pod \"kube-auth-proxy-5f99dd8ff-jf658\" (UID: \"30705649-53ef-425c-940a-f0c8d125e5d4\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" Jul 05 12:00:12.275510 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.275485 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30705649-53ef-425c-940a-f0c8d125e5d4-tmp\") pod \"kube-auth-proxy-5f99dd8ff-jf658\" (UID: \"30705649-53ef-425c-940a-f0c8d125e5d4\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" Jul 05 12:00:12.275826 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.275805 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30705649-53ef-425c-940a-f0c8d125e5d4-tls-certs\") pod \"kube-auth-proxy-5f99dd8ff-jf658\" (UID: \"30705649-53ef-425c-940a-f0c8d125e5d4\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" Jul 05 12:00:12.282611 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.282591 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52lzz\" (UniqueName: \"kubernetes.io/projected/30705649-53ef-425c-940a-f0c8d125e5d4-kube-api-access-52lzz\") pod \"kube-auth-proxy-5f99dd8ff-jf658\" (UID: \"30705649-53ef-425c-940a-f0c8d125e5d4\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" Jul 05 12:00:12.313674 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.313637 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" Jul 05 12:00:12.384674 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.384584 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:12.456038 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.456011 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658"] Jul 05 12:00:12.458086 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:00:12.458063 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30705649_53ef_425c_940a_f0c8d125e5d4.slice/crio-9c3864c4dad0af182728d3113e62f645519dab67378187d600944c58620a3a11 WatchSource:0}: Error finding container 9c3864c4dad0af182728d3113e62f645519dab67378187d600944c58620a3a11: Status 404 returned error can't find the container with id 9c3864c4dad0af182728d3113e62f645519dab67378187d600944c58620a3a11 Jul 05 12:00:12.495288 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.495247 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:12.747523 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.747422 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:12.884613 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.884584 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kc84j"] Jul 05 12:00:12.887724 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.887708 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" Jul 05 12:00:12.890950 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.890929 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-nqgs2\"" Jul 05 12:00:12.891100 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.891083 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 05 12:00:12.917486 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:12.917457 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kc84j"] Jul 05 12:00:13.019720 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.019618 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:13.079890 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.079840 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441cc4e0-f699-4028-9cdb-bcf7d8128546-tls-certs\") pod \"model-serving-api-54d89bd5fc-kc84j\" (UID: \"441cc4e0-f699-4028-9cdb-bcf7d8128546\") " pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" Jul 05 12:00:13.080062 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.079898 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jrm6\" (UniqueName: \"kubernetes.io/projected/441cc4e0-f699-4028-9cdb-bcf7d8128546-kube-api-access-2jrm6\") pod \"model-serving-api-54d89bd5fc-kc84j\" (UID: \"441cc4e0-f699-4028-9cdb-bcf7d8128546\") " pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" Jul 05 12:00:13.109043 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.109002 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" event={"ID":"30705649-53ef-425c-940a-f0c8d125e5d4","Type":"ContainerStarted","Data":"9c3864c4dad0af182728d3113e62f645519dab67378187d600944c58620a3a11"} Jul 05 12:00:13.160889 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.160845 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-czmgz"] Jul 05 12:00:13.164070 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.164050 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.165647 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.165613 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:13.166834 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.166807 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 05 12:00:13.166834 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.166817 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 05 12:00:13.167041 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.166864 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-w27d9\"" Jul 05 12:00:13.167085 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.167044 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 05 12:00:13.167126 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.167112 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 05 12:00:13.174625 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.174600 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-czmgz"] Jul 05 12:00:13.180810 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.180775 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441cc4e0-f699-4028-9cdb-bcf7d8128546-tls-certs\") pod \"model-serving-api-54d89bd5fc-kc84j\" (UID: \"441cc4e0-f699-4028-9cdb-bcf7d8128546\") " pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" Jul 05 12:00:13.180988 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.180817 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jrm6\" (UniqueName: \"kubernetes.io/projected/441cc4e0-f699-4028-9cdb-bcf7d8128546-kube-api-access-2jrm6\") pod \"model-serving-api-54d89bd5fc-kc84j\" (UID: \"441cc4e0-f699-4028-9cdb-bcf7d8128546\") " pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" Jul 05 12:00:13.183290 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.183259 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441cc4e0-f699-4028-9cdb-bcf7d8128546-tls-certs\") pod \"model-serving-api-54d89bd5fc-kc84j\" (UID: \"441cc4e0-f699-4028-9cdb-bcf7d8128546\") " pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" Jul 05 12:00:13.192119 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.192090 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jrm6\" (UniqueName: \"kubernetes.io/projected/441cc4e0-f699-4028-9cdb-bcf7d8128546-kube-api-access-2jrm6\") pod \"model-serving-api-54d89bd5fc-kc84j\" (UID: \"441cc4e0-f699-4028-9cdb-bcf7d8128546\") " pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" Jul 05 12:00:13.197830 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.197794 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" Jul 05 12:00:13.224483 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.224449 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:13.281958 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.281500 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.281958 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.281564 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/70eb429e-46e9-4597-82ec-4c3ce8461f4c-local-certs\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.281958 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.281596 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.281958 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.281637 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9pd2\" (UniqueName: \"kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-kube-api-access-q9pd2\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.281958 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.281707 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.281958 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.281764 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-cacerts\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.281958 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.281788 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-token\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.342097 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.342067 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kc84j"] Jul 05 12:00:13.344443 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:00:13.344413 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod441cc4e0_f699_4028_9cdb_bcf7d8128546.slice/crio-496ab60efa0f3f1751eb511acbc02df7b5799a86672f5e4d5efc6a204226f70c WatchSource:0}: Error finding container 496ab60efa0f3f1751eb511acbc02df7b5799a86672f5e4d5efc6a204226f70c: Status 404 returned error can't find the container with id 496ab60efa0f3f1751eb511acbc02df7b5799a86672f5e4d5efc6a204226f70c Jul 05 12:00:13.382754 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.382722 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.382943 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.382777 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/70eb429e-46e9-4597-82ec-4c3ce8461f4c-local-certs\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.382943 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.382804 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.382943 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.382845 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9pd2\" (UniqueName: \"kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-kube-api-access-q9pd2\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.382943 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.382893 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.383154 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.382950 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-cacerts\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.383154 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.382977 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-token\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.383655 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.383593 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.386118 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.386090 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.386118 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.386103 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/70eb429e-46e9-4597-82ec-4c3ce8461f4c-local-certs\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.386282 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.386099 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.386500 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.386469 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-cacerts\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.391091 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.391070 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-token\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.391206 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.391108 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9pd2\" (UniqueName: \"kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-kube-api-access-q9pd2\") pod \"istiod-openshift-gateway-7574459656-czmgz\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.474798 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.474748 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:13.620332 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:13.620304 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-czmgz"] Jul 05 12:00:13.622606 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:00:13.622579 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70eb429e_46e9_4597_82ec_4c3ce8461f4c.slice/crio-50ed55cf0bbf0aa5e6e8c1fac9114e2ce7dcbc5d9d1bbd15112a4ec38c7e6c07 WatchSource:0}: Error finding container 50ed55cf0bbf0aa5e6e8c1fac9114e2ce7dcbc5d9d1bbd15112a4ec38c7e6c07: Status 404 returned error can't find the container with id 50ed55cf0bbf0aa5e6e8c1fac9114e2ce7dcbc5d9d1bbd15112a4ec38c7e6c07 Jul 05 12:00:14.052145 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:14.052052 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:14.117013 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:14.116974 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" event={"ID":"70eb429e-46e9-4597-82ec-4c3ce8461f4c","Type":"ContainerStarted","Data":"50ed55cf0bbf0aa5e6e8c1fac9114e2ce7dcbc5d9d1bbd15112a4ec38c7e6c07"} Jul 05 12:00:14.118627 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:14.118591 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" event={"ID":"441cc4e0-f699-4028-9cdb-bcf7d8128546","Type":"ContainerStarted","Data":"496ab60efa0f3f1751eb511acbc02df7b5799a86672f5e4d5efc6a204226f70c"} Jul 05 12:00:16.135124 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:16.135087 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" event={"ID":"30705649-53ef-425c-940a-f0c8d125e5d4","Type":"ContainerStarted","Data":"65bd42a8e14c89d6f075f717a0b4cea2cbd3c60342a143627697c1f59f9753aa"} Jul 05 12:00:16.136507 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:16.136473 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" event={"ID":"441cc4e0-f699-4028-9cdb-bcf7d8128546","Type":"ContainerStarted","Data":"f62c6b77ec59b3cf284f0f4d73e4bdabb069416bdc0d477e067817a1888625ee"} Jul 05 12:00:16.136625 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:16.136524 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" Jul 05 12:00:16.153135 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:16.153087 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-jf658" podStartSLOduration=2.075903824 podStartE2EDuration="5.153061437s" totalImagesPullingTime="3.077157613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:00:16.151969919 +0000 UTC m=+416.542671674" watchObservedRunningTime="2026-07-05 12:00:16.153061437 +0000 UTC m=+416.543763200" Jul 05 12:00:16.168665 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:16.168621 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" podStartSLOduration=1.979607383 podStartE2EDuration="4.168608599s" totalImagesPullingTime="2.189001216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:00:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:00:16.166683366 +0000 UTC m=+416.557385128" watchObservedRunningTime="2026-07-05 12:00:16.168608599 +0000 UTC m=+416.559310361" Jul 05 12:00:20.424767 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:20.424728 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:22.485425 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.485387 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:22.536326 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.536296 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf"] Jul 05 12:00:22.541109 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.541085 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" Jul 05 12:00:22.541255 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.541169 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t"] Jul 05 12:00:22.544065 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.544037 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-j9srk\"" Jul 05 12:00:22.544348 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.544333 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 05 12:00:22.544590 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.544574 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" Jul 05 12:00:22.546898 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.546867 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 05 12:00:22.547135 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.546926 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-gj2r2\"" Jul 05 12:00:22.550518 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.550492 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf"] Jul 05 12:00:22.555100 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.555079 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t"] Jul 05 12:00:22.562894 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.562854 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cc93917-ad73-4c1e-83c6-0df9acb97352-cert\") pod \"kserve-controller-manager-d6f4b65b8-bg6zf\" (UID: \"5cc93917-ad73-4c1e-83c6-0df9acb97352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" Jul 05 12:00:22.562995 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.562963 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7d41648-4c6a-4218-98f9-ceb6b36ba9d3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mgw2t\" (UID: \"b7d41648-4c6a-4218-98f9-ceb6b36ba9d3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" Jul 05 12:00:22.563054 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.563006 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd4kd\" (UniqueName: \"kubernetes.io/projected/5cc93917-ad73-4c1e-83c6-0df9acb97352-kube-api-access-bd4kd\") pod \"kserve-controller-manager-d6f4b65b8-bg6zf\" (UID: \"5cc93917-ad73-4c1e-83c6-0df9acb97352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" Jul 05 12:00:22.563054 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.563037 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j72zt\" (UniqueName: \"kubernetes.io/projected/b7d41648-4c6a-4218-98f9-ceb6b36ba9d3-kube-api-access-j72zt\") pod \"llmisvc-controller-manager-777c6bd7d5-mgw2t\" (UID: \"b7d41648-4c6a-4218-98f9-ceb6b36ba9d3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" Jul 05 12:00:22.579793 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.579764 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:22.664245 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.664207 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7d41648-4c6a-4218-98f9-ceb6b36ba9d3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mgw2t\" (UID: \"b7d41648-4c6a-4218-98f9-ceb6b36ba9d3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" Jul 05 12:00:22.664245 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.664248 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd4kd\" (UniqueName: \"kubernetes.io/projected/5cc93917-ad73-4c1e-83c6-0df9acb97352-kube-api-access-bd4kd\") pod \"kserve-controller-manager-d6f4b65b8-bg6zf\" (UID: \"5cc93917-ad73-4c1e-83c6-0df9acb97352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" Jul 05 12:00:22.664504 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.664271 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j72zt\" (UniqueName: \"kubernetes.io/projected/b7d41648-4c6a-4218-98f9-ceb6b36ba9d3-kube-api-access-j72zt\") pod \"llmisvc-controller-manager-777c6bd7d5-mgw2t\" (UID: \"b7d41648-4c6a-4218-98f9-ceb6b36ba9d3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" Jul 05 12:00:22.664504 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.664332 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cc93917-ad73-4c1e-83c6-0df9acb97352-cert\") pod \"kserve-controller-manager-d6f4b65b8-bg6zf\" (UID: \"5cc93917-ad73-4c1e-83c6-0df9acb97352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" Jul 05 12:00:22.664504 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:00:22.664347 2575 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 05 12:00:22.664504 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:00:22.664445 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d41648-4c6a-4218-98f9-ceb6b36ba9d3-cert podName:b7d41648-4c6a-4218-98f9-ceb6b36ba9d3 nodeName:}" failed. No retries permitted until 2026-07-05 12:00:23.164425128 +0000 UTC m=+423.555126871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b7d41648-4c6a-4218-98f9-ceb6b36ba9d3-cert") pod "llmisvc-controller-manager-777c6bd7d5-mgw2t" (UID: "b7d41648-4c6a-4218-98f9-ceb6b36ba9d3") : secret "llmisvc-webhook-server-cert" not found Jul 05 12:00:22.664504 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:00:22.664465 2575 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 05 12:00:22.664756 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:00:22.664509 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc93917-ad73-4c1e-83c6-0df9acb97352-cert podName:5cc93917-ad73-4c1e-83c6-0df9acb97352 nodeName:}" failed. No retries permitted until 2026-07-05 12:00:23.164495294 +0000 UTC m=+423.555197034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5cc93917-ad73-4c1e-83c6-0df9acb97352-cert") pod "kserve-controller-manager-d6f4b65b8-bg6zf" (UID: "5cc93917-ad73-4c1e-83c6-0df9acb97352") : secret "kserve-webhook-server-cert" not found Jul 05 12:00:22.685240 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.685211 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j72zt\" (UniqueName: \"kubernetes.io/projected/b7d41648-4c6a-4218-98f9-ceb6b36ba9d3-kube-api-access-j72zt\") pod \"llmisvc-controller-manager-777c6bd7d5-mgw2t\" (UID: \"b7d41648-4c6a-4218-98f9-ceb6b36ba9d3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" Jul 05 12:00:22.685462 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.685439 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd4kd\" (UniqueName: \"kubernetes.io/projected/5cc93917-ad73-4c1e-83c6-0df9acb97352-kube-api-access-bd4kd\") pod \"kserve-controller-manager-d6f4b65b8-bg6zf\" (UID: \"5cc93917-ad73-4c1e-83c6-0df9acb97352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" Jul 05 12:00:22.792536 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:22.792443 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:23.032712 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:23.032675 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:23.169102 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:23.169071 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7d41648-4c6a-4218-98f9-ceb6b36ba9d3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mgw2t\" (UID: \"b7d41648-4c6a-4218-98f9-ceb6b36ba9d3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" Jul 05 12:00:23.169307 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:23.169224 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cc93917-ad73-4c1e-83c6-0df9acb97352-cert\") pod \"kserve-controller-manager-d6f4b65b8-bg6zf\" (UID: \"5cc93917-ad73-4c1e-83c6-0df9acb97352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" Jul 05 12:00:23.171677 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:23.171647 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cc93917-ad73-4c1e-83c6-0df9acb97352-cert\") pod \"kserve-controller-manager-d6f4b65b8-bg6zf\" (UID: \"5cc93917-ad73-4c1e-83c6-0df9acb97352\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" Jul 05 12:00:23.171790 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:23.171647 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7d41648-4c6a-4218-98f9-ceb6b36ba9d3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mgw2t\" (UID: \"b7d41648-4c6a-4218-98f9-ceb6b36ba9d3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" Jul 05 12:00:23.454751 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:23.454665 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" Jul 05 12:00:23.460896 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:23.460863 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" Jul 05 12:00:23.652962 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:23.652924 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf"] Jul 05 12:00:23.653976 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:23.653950 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t"] Jul 05 12:00:23.655226 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:00:23.655198 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cc93917_ad73_4c1e_83c6_0df9acb97352.slice/crio-42e561961e9131947c0bf9b4dd8fb4ecf4d3f354faa1f8795e737a3fe71cc9c1 WatchSource:0}: Error finding container 42e561961e9131947c0bf9b4dd8fb4ecf4d3f354faa1f8795e737a3fe71cc9c1: Status 404 returned error can't find the container with id 42e561961e9131947c0bf9b4dd8fb4ecf4d3f354faa1f8795e737a3fe71cc9c1 Jul 05 12:00:24.156328 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:24.156278 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:24.176317 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:24.176284 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" event={"ID":"b7d41648-4c6a-4218-98f9-ceb6b36ba9d3","Type":"ContainerStarted","Data":"4a01147bd33e888f23187bdb4a2f5d7b4994d4239b63b28dc925bd5dbc521e29"} Jul 05 12:00:24.177433 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:24.177402 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" event={"ID":"5cc93917-ad73-4c1e-83c6-0df9acb97352","Type":"ContainerStarted","Data":"42e561961e9131947c0bf9b4dd8fb4ecf4d3f354faa1f8795e737a3fe71cc9c1"} Jul 05 12:00:24.272764 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:24.272716 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:24.329962 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:24.329903 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:25.447301 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:25.447259 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:25.549817 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:25.549779 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:25.758410 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:25.758316 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:25.975297 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:25.975261 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:26.077371 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:26.077328 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:00:27.144703 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:27.144670 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-kc84j" Jul 05 12:00:27.192553 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:27.192519 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" event={"ID":"b7d41648-4c6a-4218-98f9-ceb6b36ba9d3","Type":"ContainerStarted","Data":"50498a648c1fed19ac3cafeb390803d57fceeb986017c1ac1c23d535654e8aee"} Jul 05 12:00:27.192740 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:27.192608 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" Jul 05 12:00:27.194194 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:27.194165 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" event={"ID":"5cc93917-ad73-4c1e-83c6-0df9acb97352","Type":"ContainerStarted","Data":"3f1532e2e10bbbec89422550ba08d7de05b6129b95d14123f6e64ff38548a7b3"} Jul 05 12:00:27.194327 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:27.194234 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" Jul 05 12:00:27.208606 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:27.208562 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" podStartSLOduration=2.22512842 podStartE2EDuration="5.208547555s" totalImagesPullingTime="2.983419135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:00:27.207083792 +0000 UTC m=+427.597785552" watchObservedRunningTime="2026-07-05 12:00:27.208547555 +0000 UTC m=+427.599249316" Jul 05 12:00:27.222012 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:27.221965 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" podStartSLOduration=2.242568112 podStartE2EDuration="5.221948761s" totalImagesPullingTime="2.979380649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:00:27.221449612 +0000 UTC m=+427.612151376" watchObservedRunningTime="2026-07-05 12:00:27.221948761 +0000 UTC m=+427.612650504" Jul 05 12:00:28.577995 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:28.577955 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-czmgz"] Jul 05 12:00:31.250300 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:31.250261 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 05 12:00:31.250599 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:31.250325 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 05 12:00:32.219935 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.219864 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" event={"ID":"70eb429e-46e9-4597-82ec-4c3ce8461f4c","Type":"ContainerStarted","Data":"628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439"} Jul 05 12:00:32.219935 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.219920 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" podUID="70eb429e-46e9-4597-82ec-4c3ce8461f4c" containerName="discovery" containerID="cri-o://628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439" gracePeriod=30 Jul 05 12:00:32.220217 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.220028 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:32.222064 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.222040 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:32.240176 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.240135 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" podStartSLOduration=1.615270752 podStartE2EDuration="19.240121455s" totalImagesPullingTime="17.624850703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:00:32.237823899 +0000 UTC m=+432.628525673" watchObservedRunningTime="2026-07-05 12:00:32.240121455 +0000 UTC m=+432.630823218" Jul 05 12:00:32.463576 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.463552 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:32.560232 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.560204 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9pd2\" (UniqueName: \"kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-kube-api-access-q9pd2\") pod \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " Jul 05 12:00:32.560429 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.560246 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-kubeconfig\") pod \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " Jul 05 12:00:32.560429 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.560270 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-cacerts\") pod \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " Jul 05 12:00:32.560429 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.560406 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-dns-cert\") pod \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " Jul 05 12:00:32.560600 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.560475 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-token\") pod \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " Jul 05 12:00:32.560600 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.560535 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-ca-configmap\") pod \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " Jul 05 12:00:32.560600 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.560575 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/70eb429e-46e9-4597-82ec-4c3ce8461f4c-local-certs\") pod \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\" (UID: \"70eb429e-46e9-4597-82ec-4c3ce8461f4c\") " Jul 05 12:00:32.560934 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.560908 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "70eb429e-46e9-4597-82ec-4c3ce8461f4c" (UID: "70eb429e-46e9-4597-82ec-4c3ce8461f4c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:00:32.561242 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.561222 2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-ca-configmap\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:00:32.562615 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.562577 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70eb429e-46e9-4597-82ec-4c3ce8461f4c-local-certs" (OuterVolumeSpecName: "local-certs") pod "70eb429e-46e9-4597-82ec-4c3ce8461f4c" (UID: "70eb429e-46e9-4597-82ec-4c3ce8461f4c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:00:32.562728 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.562676 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "70eb429e-46e9-4597-82ec-4c3ce8461f4c" (UID: "70eb429e-46e9-4597-82ec-4c3ce8461f4c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:00:32.562785 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.562746 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-cacerts" (OuterVolumeSpecName: "cacerts") pod "70eb429e-46e9-4597-82ec-4c3ce8461f4c" (UID: "70eb429e-46e9-4597-82ec-4c3ce8461f4c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:00:32.562785 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.562766 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-kube-api-access-q9pd2" (OuterVolumeSpecName: "kube-api-access-q9pd2") pod "70eb429e-46e9-4597-82ec-4c3ce8461f4c" (UID: "70eb429e-46e9-4597-82ec-4c3ce8461f4c"). InnerVolumeSpecName "kube-api-access-q9pd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:00:32.563006 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.562980 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "70eb429e-46e9-4597-82ec-4c3ce8461f4c" (UID: "70eb429e-46e9-4597-82ec-4c3ce8461f4c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:00:32.563166 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.563144 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-token" (OuterVolumeSpecName: "istio-token") pod "70eb429e-46e9-4597-82ec-4c3ce8461f4c" (UID: "70eb429e-46e9-4597-82ec-4c3ce8461f4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:00:32.661813 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.661780 2575 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/70eb429e-46e9-4597-82ec-4c3ce8461f4c-local-certs\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:00:32.661813 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.661809 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9pd2\" (UniqueName: \"kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-kube-api-access-q9pd2\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:00:32.661813 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.661819 2575 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-kubeconfig\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:00:32.662043 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.661828 2575 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-cacerts\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:00:32.662043 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.661839 2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-csr-dns-cert\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:00:32.662043 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:32.661852 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70eb429e-46e9-4597-82ec-4c3ce8461f4c-istio-token\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:00:33.226613 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:33.226581 2575 generic.go:356] "Generic (PLEG): container finished" podID="70eb429e-46e9-4597-82ec-4c3ce8461f4c" containerID="628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439" exitCode=0 Jul 05 12:00:33.226789 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:33.226634 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" Jul 05 12:00:33.226789 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:33.226668 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" event={"ID":"70eb429e-46e9-4597-82ec-4c3ce8461f4c","Type":"ContainerDied","Data":"628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439"} Jul 05 12:00:33.226789 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:33.226719 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-czmgz" event={"ID":"70eb429e-46e9-4597-82ec-4c3ce8461f4c","Type":"ContainerDied","Data":"50ed55cf0bbf0aa5e6e8c1fac9114e2ce7dcbc5d9d1bbd15112a4ec38c7e6c07"} Jul 05 12:00:33.226789 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:33.226740 2575 scope.go:117] "RemoveContainer" containerID="628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439" Jul 05 12:00:33.235799 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:33.235774 2575 scope.go:117] "RemoveContainer" containerID="628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439" Jul 05 12:00:33.236134 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:00:33.236113 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439\": container with ID starting with 628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439 not found: ID does not exist" containerID="628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439" Jul 05 12:00:33.236195 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:33.236143 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439"} err="failed to get container status \"628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439\": rpc error: code = NotFound desc = could not find container \"628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439\": container with ID starting with 628d36ff0db715f050cb720acd83120ddc93aac2747da06190c1b42bee90c439 not found: ID does not exist" Jul 05 12:00:33.247419 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:33.247390 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-czmgz"] Jul 05 12:00:33.251344 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:33.251311 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-czmgz"] Jul 05 12:00:34.129128 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:34.129096 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70eb429e-46e9-4597-82ec-4c3ce8461f4c" path="/var/lib/kubelet/pods/70eb429e-46e9-4597-82ec-4c3ce8461f4c/volumes" Jul 05 12:00:58.201176 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:58.201088 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mgw2t" Jul 05 12:00:58.204074 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:00:58.204053 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bg6zf" Jul 05 12:01:01.970262 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:01.970210 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:12.929314 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:12.929274 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:13.052714 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.052639 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:13.088917 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.088868 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gd4bk"] Jul 05 12:01:13.089254 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.089241 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70eb429e-46e9-4597-82ec-4c3ce8461f4c" containerName="discovery" Jul 05 12:01:13.089302 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.089256 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="70eb429e-46e9-4597-82ec-4c3ce8461f4c" containerName="discovery" Jul 05 12:01:13.089337 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.089318 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="70eb429e-46e9-4597-82ec-4c3ce8461f4c" containerName="discovery" Jul 05 12:01:13.092152 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.092133 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" Jul 05 12:01:13.094863 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.094837 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-dwh2j\"" Jul 05 12:01:13.095011 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.094960 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 05 12:01:13.096243 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.096211 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 05 12:01:13.101847 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.101826 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gd4bk"] Jul 05 12:01:13.143650 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.143597 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:13.197029 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.196948 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wtn8\" (UniqueName: \"kubernetes.io/projected/86d4b9d9-f26b-4ca3-a3da-792af3c592ae-kube-api-access-5wtn8\") pod \"kuadrant-operator-catalog-gd4bk\" (UID: \"86d4b9d9-f26b-4ca3-a3da-792af3c592ae\") " pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" Jul 05 12:01:13.297847 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.297814 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wtn8\" (UniqueName: \"kubernetes.io/projected/86d4b9d9-f26b-4ca3-a3da-792af3c592ae-kube-api-access-5wtn8\") pod \"kuadrant-operator-catalog-gd4bk\" (UID: \"86d4b9d9-f26b-4ca3-a3da-792af3c592ae\") " pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" Jul 05 12:01:13.305419 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.305391 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wtn8\" (UniqueName: \"kubernetes.io/projected/86d4b9d9-f26b-4ca3-a3da-792af3c592ae-kube-api-access-5wtn8\") pod \"kuadrant-operator-catalog-gd4bk\" (UID: \"86d4b9d9-f26b-4ca3-a3da-792af3c592ae\") " pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" Jul 05 12:01:13.402847 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.402812 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" Jul 05 12:01:13.442725 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.442688 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gd4bk"] Jul 05 12:01:13.532622 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.532591 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gd4bk"] Jul 05 12:01:13.535511 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:01:13.535486 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d4b9d9_f26b_4ca3_a3da_792af3c592ae.slice/crio-7fefe4ddef89e9be289be3473fce0fc653bb40c94279a4bbf03907530c08a5ff WatchSource:0}: Error finding container 7fefe4ddef89e9be289be3473fce0fc653bb40c94279a4bbf03907530c08a5ff: Status 404 returned error can't find the container with id 7fefe4ddef89e9be289be3473fce0fc653bb40c94279a4bbf03907530c08a5ff Jul 05 12:01:13.647662 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.647628 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k4gb8"] Jul 05 12:01:13.652109 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.652091 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" Jul 05 12:01:13.657236 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.657213 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k4gb8"] Jul 05 12:01:13.699919 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.699889 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwnkl\" (UniqueName: \"kubernetes.io/projected/1d27ba7f-71e4-4e6c-b457-f9f4b461c4a7-kube-api-access-wwnkl\") pod \"kuadrant-operator-catalog-k4gb8\" (UID: \"1d27ba7f-71e4-4e6c-b457-f9f4b461c4a7\") " pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" Jul 05 12:01:13.800845 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.800759 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwnkl\" (UniqueName: \"kubernetes.io/projected/1d27ba7f-71e4-4e6c-b457-f9f4b461c4a7-kube-api-access-wwnkl\") pod \"kuadrant-operator-catalog-k4gb8\" (UID: \"1d27ba7f-71e4-4e6c-b457-f9f4b461c4a7\") " pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" Jul 05 12:01:13.809287 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.809265 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwnkl\" (UniqueName: \"kubernetes.io/projected/1d27ba7f-71e4-4e6c-b457-f9f4b461c4a7-kube-api-access-wwnkl\") pod \"kuadrant-operator-catalog-k4gb8\" (UID: \"1d27ba7f-71e4-4e6c-b457-f9f4b461c4a7\") " pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" Jul 05 12:01:13.868533 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.868500 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:13.962693 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:13.962644 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" Jul 05 12:01:14.080152 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:14.080087 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:14.254000 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:14.253966 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k4gb8"] Jul 05 12:01:14.282598 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:01:14.282566 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d27ba7f_71e4_4e6c_b457_f9f4b461c4a7.slice/crio-4a01206d7c7d59fcd8c6bb5d6062b0172b999e1665de358c731898502f9f9c26 WatchSource:0}: Error finding container 4a01206d7c7d59fcd8c6bb5d6062b0172b999e1665de358c731898502f9f9c26: Status 404 returned error can't find the container with id 4a01206d7c7d59fcd8c6bb5d6062b0172b999e1665de358c731898502f9f9c26 Jul 05 12:01:14.403512 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:14.403425 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" event={"ID":"86d4b9d9-f26b-4ca3-a3da-792af3c592ae","Type":"ContainerStarted","Data":"7fefe4ddef89e9be289be3473fce0fc653bb40c94279a4bbf03907530c08a5ff"} Jul 05 12:01:14.404656 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:14.404627 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" event={"ID":"1d27ba7f-71e4-4e6c-b457-f9f4b461c4a7","Type":"ContainerStarted","Data":"4a01206d7c7d59fcd8c6bb5d6062b0172b999e1665de358c731898502f9f9c26"} Jul 05 12:01:16.414963 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:16.414925 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" event={"ID":"86d4b9d9-f26b-4ca3-a3da-792af3c592ae","Type":"ContainerStarted","Data":"d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86"} Jul 05 12:01:16.415394 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:16.414974 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" podUID="86d4b9d9-f26b-4ca3-a3da-792af3c592ae" containerName="registry-server" containerID="cri-o://d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86" gracePeriod=2 Jul 05 12:01:16.416512 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:16.416475 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" event={"ID":"1d27ba7f-71e4-4e6c-b457-f9f4b461c4a7","Type":"ContainerStarted","Data":"831769266d68589275c5465f6bc203ce99423381c1390d19ceb9c6c5f0d4e580"} Jul 05 12:01:16.431011 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:16.430960 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" podStartSLOduration=1.4506676459999999 podStartE2EDuration="3.430939348s" totalImagesPullingTime="1.980271702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:01:16.429510796 +0000 UTC m=+476.820212569" watchObservedRunningTime="2026-07-05 12:01:16.430939348 +0000 UTC m=+476.821641112" Jul 05 12:01:16.443486 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:16.443440 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" podStartSLOduration=2.207852059 podStartE2EDuration="3.443425453s" totalImagesPullingTime="1.235573394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:01:16.443107343 +0000 UTC m=+476.833809107" watchObservedRunningTime="2026-07-05 12:01:16.443425453 +0000 UTC m=+476.834127214" Jul 05 12:01:16.661155 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:16.661124 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" Jul 05 12:01:16.724393 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:16.724303 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wtn8\" (UniqueName: \"kubernetes.io/projected/86d4b9d9-f26b-4ca3-a3da-792af3c592ae-kube-api-access-5wtn8\") pod \"86d4b9d9-f26b-4ca3-a3da-792af3c592ae\" (UID: \"86d4b9d9-f26b-4ca3-a3da-792af3c592ae\") " Jul 05 12:01:16.726451 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:16.726426 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86d4b9d9-f26b-4ca3-a3da-792af3c592ae-kube-api-access-5wtn8" (OuterVolumeSpecName: "kube-api-access-5wtn8") pod "86d4b9d9-f26b-4ca3-a3da-792af3c592ae" (UID: "86d4b9d9-f26b-4ca3-a3da-792af3c592ae"). InnerVolumeSpecName "kube-api-access-5wtn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:01:16.825386 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:16.825341 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wtn8\" (UniqueName: \"kubernetes.io/projected/86d4b9d9-f26b-4ca3-a3da-792af3c592ae-kube-api-access-5wtn8\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:01:17.086123 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:17.086085 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:17.422920 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:17.422806 2575 generic.go:356] "Generic (PLEG): container finished" podID="86d4b9d9-f26b-4ca3-a3da-792af3c592ae" containerID="d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86" exitCode=0 Jul 05 12:01:17.422920 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:17.422850 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" Jul 05 12:01:17.423431 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:17.422913 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" event={"ID":"86d4b9d9-f26b-4ca3-a3da-792af3c592ae","Type":"ContainerDied","Data":"d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86"} Jul 05 12:01:17.423431 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:17.422971 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gd4bk" event={"ID":"86d4b9d9-f26b-4ca3-a3da-792af3c592ae","Type":"ContainerDied","Data":"7fefe4ddef89e9be289be3473fce0fc653bb40c94279a4bbf03907530c08a5ff"} Jul 05 12:01:17.423431 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:17.422997 2575 scope.go:117] "RemoveContainer" containerID="d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86" Jul 05 12:01:17.432490 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:17.432471 2575 scope.go:117] "RemoveContainer" containerID="d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86" Jul 05 12:01:17.432723 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:01:17.432704 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86\": container with ID starting with d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86 not found: ID does not exist" containerID="d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86" Jul 05 12:01:17.432776 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:17.432738 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86"} err="failed to get container status \"d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86\": rpc error: code = NotFound desc = could not find container \"d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86\": container with ID starting with d24d975d0473cdf70410f703a822d63d5375be5b4b0c0215d5da991743d02b86 not found: ID does not exist" Jul 05 12:01:17.444508 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:17.444469 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gd4bk"] Jul 05 12:01:17.446200 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:17.446180 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gd4bk"] Jul 05 12:01:18.128275 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:18.128244 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86d4b9d9-f26b-4ca3-a3da-792af3c592ae" path="/var/lib/kubelet/pods/86d4b9d9-f26b-4ca3-a3da-792af3c592ae/volumes" Jul 05 12:01:23.962802 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:23.962755 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" Jul 05 12:01:23.963324 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:23.962829 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" Jul 05 12:01:23.985353 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:23.985324 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" Jul 05 12:01:24.480037 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:24.480009 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-k4gb8" Jul 05 12:01:28.398425 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:28.398388 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:28.499219 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:28.499178 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:28.546535 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:28.546496 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:41.382004 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:41.381965 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:41.980370 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:41.980314 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:45.436392 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.436361 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tpw94"] Jul 05 12:01:45.436829 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.436692 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86d4b9d9-f26b-4ca3-a3da-792af3c592ae" containerName="registry-server" Jul 05 12:01:45.436829 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.436702 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="86d4b9d9-f26b-4ca3-a3da-792af3c592ae" containerName="registry-server" Jul 05 12:01:45.436829 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.436768 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="86d4b9d9-f26b-4ca3-a3da-792af3c592ae" containerName="registry-server" Jul 05 12:01:45.441301 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.441283 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tpw94" Jul 05 12:01:45.446028 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.446007 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-n6mmf\"" Jul 05 12:01:45.455734 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.455709 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tpw94"] Jul 05 12:01:45.576603 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.576569 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx42p\" (UniqueName: \"kubernetes.io/projected/89bb08b7-fe0c-4536-b29d-a046f2534a65-kube-api-access-dx42p\") pod \"authorino-operator-75cc8d5cb7-tpw94\" (UID: \"89bb08b7-fe0c-4536-b29d-a046f2534a65\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tpw94" Jul 05 12:01:45.677911 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.677853 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx42p\" (UniqueName: \"kubernetes.io/projected/89bb08b7-fe0c-4536-b29d-a046f2534a65-kube-api-access-dx42p\") pod \"authorino-operator-75cc8d5cb7-tpw94\" (UID: \"89bb08b7-fe0c-4536-b29d-a046f2534a65\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tpw94" Jul 05 12:01:45.689495 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.689429 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx42p\" (UniqueName: \"kubernetes.io/projected/89bb08b7-fe0c-4536-b29d-a046f2534a65-kube-api-access-dx42p\") pod \"authorino-operator-75cc8d5cb7-tpw94\" (UID: \"89bb08b7-fe0c-4536-b29d-a046f2534a65\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tpw94" Jul 05 12:01:45.751314 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.751279 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tpw94" Jul 05 12:01:45.890036 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:45.890009 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tpw94"] Jul 05 12:01:45.892325 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:01:45.892296 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89bb08b7_fe0c_4536_b29d_a046f2534a65.slice/crio-8455015e19f85021f284974dc1e23a08008fadf55d3e5e4310158dc6b141d459 WatchSource:0}: Error finding container 8455015e19f85021f284974dc1e23a08008fadf55d3e5e4310158dc6b141d459: Status 404 returned error can't find the container with id 8455015e19f85021f284974dc1e23a08008fadf55d3e5e4310158dc6b141d459 Jul 05 12:01:46.560182 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:46.560144 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tpw94" event={"ID":"89bb08b7-fe0c-4536-b29d-a046f2534a65","Type":"ContainerStarted","Data":"8455015e19f85021f284974dc1e23a08008fadf55d3e5e4310158dc6b141d459"} Jul 05 12:01:46.805379 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:46.805340 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:48.570866 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.570829 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tpw94" event={"ID":"89bb08b7-fe0c-4536-b29d-a046f2534a65","Type":"ContainerStarted","Data":"05c711148657983e721feacc70b0b6cf9b116e1cfc5186135ef3271c95f4cd68"} Jul 05 12:01:48.571307 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.570939 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tpw94" Jul 05 12:01:48.590962 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.590903 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tpw94" podStartSLOduration=1.275405182 podStartE2EDuration="3.590886045s" totalImagesPullingTime="2.315480863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:01:48.589441515 +0000 UTC m=+508.980143288" watchObservedRunningTime="2026-07-05 12:01:48.590886045 +0000 UTC m=+508.981587798" Jul 05 12:01:48.634897 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.634856 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42"] Jul 05 12:01:48.638274 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.638255 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42" Jul 05 12:01:48.641154 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.641127 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qpxmr\"" Jul 05 12:01:48.641309 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.641171 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 05 12:01:48.648344 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.648318 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42"] Jul 05 12:01:48.704227 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.704189 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z7d2\" (UniqueName: \"kubernetes.io/projected/a22ce71e-6388-4407-adaa-0fc36fda26be-kube-api-access-7z7d2\") pod \"dns-operator-controller-manager-864dd78c4c-hlp42\" (UID: \"a22ce71e-6388-4407-adaa-0fc36fda26be\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42" Jul 05 12:01:48.804765 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.804729 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7d2\" (UniqueName: \"kubernetes.io/projected/a22ce71e-6388-4407-adaa-0fc36fda26be-kube-api-access-7z7d2\") pod \"dns-operator-controller-manager-864dd78c4c-hlp42\" (UID: \"a22ce71e-6388-4407-adaa-0fc36fda26be\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42" Jul 05 12:01:48.814936 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.814910 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z7d2\" (UniqueName: \"kubernetes.io/projected/a22ce71e-6388-4407-adaa-0fc36fda26be-kube-api-access-7z7d2\") pod \"dns-operator-controller-manager-864dd78c4c-hlp42\" (UID: \"a22ce71e-6388-4407-adaa-0fc36fda26be\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42" Jul 05 12:01:48.825110 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.825040 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:48.950125 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:48.950090 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42" Jul 05 12:01:49.087059 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:49.087033 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42"] Jul 05 12:01:49.089585 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:01:49.089549 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda22ce71e_6388_4407_adaa_0fc36fda26be.slice/crio-c3acf7a47679d41e48c93a1f9e8ae70d6625b31ba3946059636b94590b92b806 WatchSource:0}: Error finding container c3acf7a47679d41e48c93a1f9e8ae70d6625b31ba3946059636b94590b92b806: Status 404 returned error can't find the container with id c3acf7a47679d41e48c93a1f9e8ae70d6625b31ba3946059636b94590b92b806 Jul 05 12:01:49.576469 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:49.576427 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42" event={"ID":"a22ce71e-6388-4407-adaa-0fc36fda26be","Type":"ContainerStarted","Data":"c3acf7a47679d41e48c93a1f9e8ae70d6625b31ba3946059636b94590b92b806"} Jul 05 12:01:49.791271 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:49.791232 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:50.611080 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:50.611039 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:51.587927 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:51.587889 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42" event={"ID":"a22ce71e-6388-4407-adaa-0fc36fda26be","Type":"ContainerStarted","Data":"b6958969f48f7614b0542d46caceea0d11ed92b005df57e1f61514a25c54aa25"} Jul 05 12:01:51.588109 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:51.588022 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42" Jul 05 12:01:51.604927 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:51.604853 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42" podStartSLOduration=1.818550101 podStartE2EDuration="3.604835958s" totalImagesPullingTime="1.786285857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:01:51.60343677 +0000 UTC m=+511.994138533" watchObservedRunningTime="2026-07-05 12:01:51.604835958 +0000 UTC m=+511.995537720" Jul 05 12:01:52.183224 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:52.183183 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm"] Jul 05 12:01:52.189280 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:52.189237 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:01:52.192437 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:52.192413 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6mm49\"" Jul 05 12:01:52.196189 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:52.196164 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm"] Jul 05 12:01:52.231791 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:52.231754 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wswn\" (UniqueName: \"kubernetes.io/projected/07f18f55-f016-4f18-8d15-cf6773eca1a8-kube-api-access-2wswn\") pod \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" (UID: \"07f18f55-f016-4f18-8d15-cf6773eca1a8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:01:52.332837 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:52.332800 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wswn\" (UniqueName: \"kubernetes.io/projected/07f18f55-f016-4f18-8d15-cf6773eca1a8-kube-api-access-2wswn\") pod \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" (UID: \"07f18f55-f016-4f18-8d15-cf6773eca1a8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:01:52.340686 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:52.340659 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wswn\" (UniqueName: \"kubernetes.io/projected/07f18f55-f016-4f18-8d15-cf6773eca1a8-kube-api-access-2wswn\") pod \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" (UID: \"07f18f55-f016-4f18-8d15-cf6773eca1a8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:01:52.501497 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:52.501401 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:01:52.640400 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:52.640370 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm"] Jul 05 12:01:52.643924 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:01:52.643898 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f18f55_f016_4f18_8d15_cf6773eca1a8.slice/crio-3703fbbbb31b85fcbda4d0110380a8c9a492679e2e4ec21ac2566b6a017b7114 WatchSource:0}: Error finding container 3703fbbbb31b85fcbda4d0110380a8c9a492679e2e4ec21ac2566b6a017b7114: Status 404 returned error can't find the container with id 3703fbbbb31b85fcbda4d0110380a8c9a492679e2e4ec21ac2566b6a017b7114 Jul 05 12:01:52.794267 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:52.794159 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:53.257259 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:53.257226 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:53.600145 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:53.600106 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" event={"ID":"07f18f55-f016-4f18-8d15-cf6773eca1a8","Type":"ContainerStarted","Data":"3703fbbbb31b85fcbda4d0110380a8c9a492679e2e4ec21ac2566b6a017b7114"} Jul 05 12:01:54.607304 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:54.607210 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" event={"ID":"07f18f55-f016-4f18-8d15-cf6773eca1a8","Type":"ContainerStarted","Data":"2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57"} Jul 05 12:01:54.607654 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:54.607375 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:01:54.633531 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:54.633479 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" podStartSLOduration=0.961666538 podStartE2EDuration="2.633465829s" totalImagesPullingTime="1.671799291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:01:54.632008948 +0000 UTC m=+515.022710711" watchObservedRunningTime="2026-07-05 12:01:54.633465829 +0000 UTC m=+515.024167590" Jul 05 12:01:56.447806 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:56.447766 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:56.809358 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:56.809324 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg"] Jul 05 12:01:56.812681 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:56.812665 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:01:56.818699 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:56.818675 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r799m\"" Jul 05 12:01:56.827222 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:56.827195 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg"] Jul 05 12:01:56.869033 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:56.869000 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzlxl\" (UniqueName: \"kubernetes.io/projected/f0e4129a-e733-4db4-869f-9bfaab84a1bd-kube-api-access-pzlxl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" (UID: \"f0e4129a-e733-4db4-869f-9bfaab84a1bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:01:56.869033 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:56.869033 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0e4129a-e733-4db4-869f-9bfaab84a1bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" (UID: \"f0e4129a-e733-4db4-869f-9bfaab84a1bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:01:56.970240 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:56.970201 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzlxl\" (UniqueName: \"kubernetes.io/projected/f0e4129a-e733-4db4-869f-9bfaab84a1bd-kube-api-access-pzlxl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" (UID: \"f0e4129a-e733-4db4-869f-9bfaab84a1bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:01:56.970240 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:56.970241 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0e4129a-e733-4db4-869f-9bfaab84a1bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" (UID: \"f0e4129a-e733-4db4-869f-9bfaab84a1bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:01:56.970666 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:56.970644 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0e4129a-e733-4db4-869f-9bfaab84a1bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" (UID: \"f0e4129a-e733-4db4-869f-9bfaab84a1bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:01:56.981274 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:56.981243 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzlxl\" (UniqueName: \"kubernetes.io/projected/f0e4129a-e733-4db4-869f-9bfaab84a1bd-kube-api-access-pzlxl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" (UID: \"f0e4129a-e733-4db4-869f-9bfaab84a1bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:01:57.122752 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:57.122667 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:01:57.262420 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:57.262389 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg"] Jul 05 12:01:57.264794 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:01:57.264758 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e4129a_e733_4db4_869f_9bfaab84a1bd.slice/crio-f4bd3ae2a77246cbfe96d547454f75e46f27f1f471d4eea6e84e9431d6401aa6 WatchSource:0}: Error finding container f4bd3ae2a77246cbfe96d547454f75e46f27f1f471d4eea6e84e9431d6401aa6: Status 404 returned error can't find the container with id f4bd3ae2a77246cbfe96d547454f75e46f27f1f471d4eea6e84e9431d6401aa6 Jul 05 12:01:57.621490 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:57.621453 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" event={"ID":"f0e4129a-e733-4db4-869f-9bfaab84a1bd","Type":"ContainerStarted","Data":"f4bd3ae2a77246cbfe96d547454f75e46f27f1f471d4eea6e84e9431d6401aa6"} Jul 05 12:01:58.399398 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:58.399364 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:58.502565 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:58.502529 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:58.804857 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:58.804821 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:58.902952 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:58.902917 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:59.409890 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:59.409833 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:01:59.578666 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:01:59.578637 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tpw94" Jul 05 12:02:02.187233 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:02.187189 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:02.237085 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:02.237048 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:02.596326 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:02.596290 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hlp42" Jul 05 12:02:03.657371 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:03.657335 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" event={"ID":"f0e4129a-e733-4db4-869f-9bfaab84a1bd","Type":"ContainerStarted","Data":"30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980"} Jul 05 12:02:03.657834 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:03.657416 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:03.677235 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:03.677156 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" podStartSLOduration=2.141936682 podStartE2EDuration="7.67713763s" totalImagesPullingTime="5.535200948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:01:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:02:03.675428803 +0000 UTC m=+524.066130575" watchObservedRunningTime="2026-07-05 12:02:03.67713763 +0000 UTC m=+524.067839392" Jul 05 12:02:05.614192 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:05.614159 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:05.674832 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:05.674786 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:06.164471 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:06.164432 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:06.400964 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:06.400923 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:06.501829 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:06.501734 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:09.517724 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:09.517685 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:09.618388 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:09.618356 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:14.664418 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:14.664338 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:16.296548 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.296516 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg"] Jul 05 12:02:16.296985 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.296685 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" containerName="manager" containerID="cri-o://30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980" gracePeriod=2 Jul 05 12:02:16.315752 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.315690 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg"] Jul 05 12:02:16.322907 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.322849 2575 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f0e4129a-e733-4db4-869f-9bfaab84a1bd\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\": pods \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:16.333552 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.332789 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm"] Jul 05 12:02:16.333552 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.333054 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" containerName="manager" containerID="cri-o://2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57" gracePeriod=2 Jul 05 12:02:16.339200 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.337073 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:16.348892 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.348467 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm"] Jul 05 12:02:16.355688 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.355521 2575 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"07f18f55-f016-4f18-8d15-cf6773eca1a8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-sbtgm\": pods \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:16.357957 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.357928 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs"] Jul 05 12:02:16.358282 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.358263 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" containerName="manager" Jul 05 12:02:16.358282 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.358280 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" containerName="manager" Jul 05 12:02:16.358441 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.358320 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" containerName="manager" Jul 05 12:02:16.358441 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.358329 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" containerName="manager" Jul 05 12:02:16.358539 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.358435 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" containerName="manager" Jul 05 12:02:16.358539 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.358500 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" containerName="manager" Jul 05 12:02:16.363814 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.363797 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs" Jul 05 12:02:16.366117 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.366086 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:16.368649 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.368616 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:16.370109 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.370087 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs"] Jul 05 12:02:16.440647 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.440608 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22tw4\" (UniqueName: \"kubernetes.io/projected/bb0dab62-164f-4b62-9c10-14d76ede33ff-kube-api-access-22tw4\") pod \"limitador-operator-controller-manager-859d7b78bd-k77gs\" (UID: \"bb0dab62-164f-4b62-9c10-14d76ede33ff\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs" Jul 05 12:02:16.541569 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.541533 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22tw4\" (UniqueName: \"kubernetes.io/projected/bb0dab62-164f-4b62-9c10-14d76ede33ff-kube-api-access-22tw4\") pod \"limitador-operator-controller-manager-859d7b78bd-k77gs\" (UID: \"bb0dab62-164f-4b62-9c10-14d76ede33ff\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs" Jul 05 12:02:16.552524 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.552501 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22tw4\" (UniqueName: \"kubernetes.io/projected/bb0dab62-164f-4b62-9c10-14d76ede33ff-kube-api-access-22tw4\") pod \"limitador-operator-controller-manager-859d7b78bd-k77gs\" (UID: \"bb0dab62-164f-4b62-9c10-14d76ede33ff\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs" Jul 05 12:02:16.593751 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.593729 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:16.596177 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.596149 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:16.596505 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.596490 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:16.598317 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.598292 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:16.600311 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.600290 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:16.602246 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.602224 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:16.722847 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.722811 2575 generic.go:356] "Generic (PLEG): container finished" podID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" containerID="30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980" exitCode=0 Jul 05 12:02:16.723071 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.722859 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:16.723071 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.722918 2575 scope.go:117] "RemoveContainer" containerID="30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980" Jul 05 12:02:16.724409 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.724380 2575 generic.go:356] "Generic (PLEG): container finished" podID="07f18f55-f016-4f18-8d15-cf6773eca1a8" containerID="2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57" exitCode=0 Jul 05 12:02:16.724517 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.724439 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:16.725483 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.725459 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:16.727584 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.727555 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:16.729625 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.729604 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:16.731653 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.731636 2575 scope.go:117] "RemoveContainer" containerID="30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980" Jul 05 12:02:16.731722 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.731685 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:16.731928 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.731909 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980\": container with ID starting with 30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980 not found: ID does not exist" containerID="30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980" Jul 05 12:02:16.731975 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.731935 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980"} err="failed to get container status \"30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980\": rpc error: code = NotFound desc = could not find container \"30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980\": container with ID starting with 30677239b253e157ce663c9a17ffd90d638e3af47017e0a04d34fb753d837980 not found: ID does not exist" Jul 05 12:02:16.731975 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.731953 2575 scope.go:117] "RemoveContainer" containerID="2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57" Jul 05 12:02:16.739313 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.739296 2575 scope.go:117] "RemoveContainer" containerID="2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57" Jul 05 12:02:16.739543 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:16.739528 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57\": container with ID starting with 2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57 not found: ID does not exist" containerID="2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57" Jul 05 12:02:16.739583 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.739549 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57"} err="failed to get container status \"2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57\": rpc error: code = NotFound desc = could not find container \"2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57\": container with ID starting with 2a4acfa610014ddeb22793366aa9049f403db1c2b849240adfbb3b4362d0cb57 not found: ID does not exist" Jul 05 12:02:16.742801 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.742780 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs" Jul 05 12:02:16.742943 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.742928 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzlxl\" (UniqueName: \"kubernetes.io/projected/f0e4129a-e733-4db4-869f-9bfaab84a1bd-kube-api-access-pzlxl\") pod \"f0e4129a-e733-4db4-869f-9bfaab84a1bd\" (UID: \"f0e4129a-e733-4db4-869f-9bfaab84a1bd\") " Jul 05 12:02:16.742993 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.742982 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wswn\" (UniqueName: \"kubernetes.io/projected/07f18f55-f016-4f18-8d15-cf6773eca1a8-kube-api-access-2wswn\") pod \"07f18f55-f016-4f18-8d15-cf6773eca1a8\" (UID: \"07f18f55-f016-4f18-8d15-cf6773eca1a8\") " Jul 05 12:02:16.743064 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.743054 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0e4129a-e733-4db4-869f-9bfaab84a1bd-extensions-socket-volume\") pod \"f0e4129a-e733-4db4-869f-9bfaab84a1bd\" (UID: \"f0e4129a-e733-4db4-869f-9bfaab84a1bd\") " Jul 05 12:02:16.743496 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.743475 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e4129a-e733-4db4-869f-9bfaab84a1bd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f0e4129a-e733-4db4-869f-9bfaab84a1bd" (UID: "f0e4129a-e733-4db4-869f-9bfaab84a1bd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:02:16.745064 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.745043 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f18f55-f016-4f18-8d15-cf6773eca1a8-kube-api-access-2wswn" (OuterVolumeSpecName: "kube-api-access-2wswn") pod "07f18f55-f016-4f18-8d15-cf6773eca1a8" (UID: "07f18f55-f016-4f18-8d15-cf6773eca1a8"). InnerVolumeSpecName "kube-api-access-2wswn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:02:16.745064 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.745054 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e4129a-e733-4db4-869f-9bfaab84a1bd-kube-api-access-pzlxl" (OuterVolumeSpecName: "kube-api-access-pzlxl") pod "f0e4129a-e733-4db4-869f-9bfaab84a1bd" (UID: "f0e4129a-e733-4db4-869f-9bfaab84a1bd"). InnerVolumeSpecName "kube-api-access-pzlxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:02:16.844463 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.844426 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0e4129a-e733-4db4-869f-9bfaab84a1bd-extensions-socket-volume\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:02:16.844463 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.844459 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzlxl\" (UniqueName: \"kubernetes.io/projected/f0e4129a-e733-4db4-869f-9bfaab84a1bd-kube-api-access-pzlxl\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:02:16.844463 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.844472 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wswn\" (UniqueName: \"kubernetes.io/projected/07f18f55-f016-4f18-8d15-cf6773eca1a8-kube-api-access-2wswn\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:02:16.883940 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:16.883905 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs"] Jul 05 12:02:16.886193 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:02:16.886165 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0dab62_164f_4b62_9c10_14d76ede33ff.slice/crio-655bb351132168bad273541094a3d0525e65672bd709edfbdd36459c55ffea0f WatchSource:0}: Error finding container 655bb351132168bad273541094a3d0525e65672bd709edfbdd36459c55ffea0f: Status 404 returned error can't find the container with id 655bb351132168bad273541094a3d0525e65672bd709edfbdd36459c55ffea0f Jul 05 12:02:17.036759 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:17.036723 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:17.039378 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:17.039352 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:17.042058 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:17.042026 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:17.044966 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:17.044930 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:17.353289 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:17.353257 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:17.454082 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:17.454035 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:17.732198 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:17.732099 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs" event={"ID":"bb0dab62-164f-4b62-9c10-14d76ede33ff","Type":"ContainerStarted","Data":"75cdc94ae2cbf97918593d01e7d8363d60c29467eacbb2eaa160d4835169e2ed"} Jul 05 12:02:17.732198 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:17.732158 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs" event={"ID":"bb0dab62-164f-4b62-9c10-14d76ede33ff","Type":"ContainerStarted","Data":"655bb351132168bad273541094a3d0525e65672bd709edfbdd36459c55ffea0f"} Jul 05 12:02:17.732420 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:17.732201 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs" Jul 05 12:02:17.751260 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:17.751208 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs" podStartSLOduration=1.751191604 podStartE2EDuration="1.751191604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:02:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:02:17.749468158 +0000 UTC m=+538.140169921" watchObservedRunningTime="2026-07-05 12:02:17.751191604 +0000 UTC m=+538.141893366" Jul 05 12:02:17.751643 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:17.751617 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p7zfg\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p7zfg" Jul 05 12:02:17.753766 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:02:17.753741 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sbtgm\" is forbidden: User \"system:node:ip-10-0-142-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-207.ec2.internal' and this object" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbtgm" Jul 05 12:02:17.776614 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:17.776584 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:17.868732 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:17.868693 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:18.129270 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:18.129239 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07f18f55-f016-4f18-8d15-cf6773eca1a8" path="/var/lib/kubelet/pods/07f18f55-f016-4f18-8d15-cf6773eca1a8/volumes" Jul 05 12:02:18.129592 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:18.129580 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e4129a-e733-4db4-869f-9bfaab84a1bd" path="/var/lib/kubelet/pods/f0e4129a-e733-4db4-869f-9bfaab84a1bd/volumes" Jul 05 12:02:20.340195 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:20.340088 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:20.440417 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:20.440366 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:25.545906 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:25.545836 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:25.647328 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:25.647298 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:28.740287 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:28.740258 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k77gs" Jul 05 12:02:28.954921 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:28.954850 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:29.054466 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:29.054428 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:29.155357 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:29.155322 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:02:46.794602 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:46.794561 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6"] Jul 05 12:02:46.797817 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:46.797799 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:02:46.800864 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:46.800847 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r799m\"" Jul 05 12:02:46.811494 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:46.811469 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6"] Jul 05 12:02:46.895361 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:46.895326 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84674c0b-aeaa-45df-8e5e-d8f87a080c54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4spd6\" (UID: \"84674c0b-aeaa-45df-8e5e-d8f87a080c54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:02:46.895531 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:46.895451 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jnf4\" (UniqueName: \"kubernetes.io/projected/84674c0b-aeaa-45df-8e5e-d8f87a080c54-kube-api-access-4jnf4\") pod \"kuadrant-operator-controller-manager-86878c769f-4spd6\" (UID: \"84674c0b-aeaa-45df-8e5e-d8f87a080c54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:02:46.996453 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:46.996410 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84674c0b-aeaa-45df-8e5e-d8f87a080c54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4spd6\" (UID: \"84674c0b-aeaa-45df-8e5e-d8f87a080c54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:02:46.996625 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:46.996487 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jnf4\" (UniqueName: \"kubernetes.io/projected/84674c0b-aeaa-45df-8e5e-d8f87a080c54-kube-api-access-4jnf4\") pod \"kuadrant-operator-controller-manager-86878c769f-4spd6\" (UID: \"84674c0b-aeaa-45df-8e5e-d8f87a080c54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:02:46.996841 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:46.996817 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84674c0b-aeaa-45df-8e5e-d8f87a080c54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4spd6\" (UID: \"84674c0b-aeaa-45df-8e5e-d8f87a080c54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:02:47.006816 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:47.006775 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jnf4\" (UniqueName: \"kubernetes.io/projected/84674c0b-aeaa-45df-8e5e-d8f87a080c54-kube-api-access-4jnf4\") pod \"kuadrant-operator-controller-manager-86878c769f-4spd6\" (UID: \"84674c0b-aeaa-45df-8e5e-d8f87a080c54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:02:47.107519 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:47.107476 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:02:47.242316 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:47.242287 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6"] Jul 05 12:02:47.244678 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:02:47.244649 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84674c0b_aeaa_45df_8e5e_d8f87a080c54.slice/crio-996ac4ce058cedef40673e33aaecf7808043b4ef863e77e6f40f05a64e123130 WatchSource:0}: Error finding container 996ac4ce058cedef40673e33aaecf7808043b4ef863e77e6f40f05a64e123130: Status 404 returned error can't find the container with id 996ac4ce058cedef40673e33aaecf7808043b4ef863e77e6f40f05a64e123130 Jul 05 12:02:47.875109 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:47.875070 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" event={"ID":"84674c0b-aeaa-45df-8e5e-d8f87a080c54","Type":"ContainerStarted","Data":"4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9"} Jul 05 12:02:47.875501 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:47.875126 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" event={"ID":"84674c0b-aeaa-45df-8e5e-d8f87a080c54","Type":"ContainerStarted","Data":"996ac4ce058cedef40673e33aaecf7808043b4ef863e77e6f40f05a64e123130"} Jul 05 12:02:47.875501 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:47.875156 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:02:47.900895 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:47.900833 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" podStartSLOduration=1.9008183669999998 podStartE2EDuration="1.900818367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:02:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:02:47.898942074 +0000 UTC m=+568.289643840" watchObservedRunningTime="2026-07-05 12:02:47.900818367 +0000 UTC m=+568.291520130" Jul 05 12:02:58.885609 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:02:58.885575 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:03:02.972070 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:02.972024 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:04.160532 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:04.160496 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:07.128650 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.128611 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:07.235223 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.235175 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:07.286160 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.286120 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:07.333971 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.333936 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:03:07.338899 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.338856 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" Jul 05 12:03:07.341673 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.341646 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-r5gq7\"" Jul 05 12:03:07.341796 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.341695 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 05 12:03:07.345326 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.345305 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:03:07.370272 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.370246 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:03:07.468002 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.467918 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c44qz\" (UniqueName: \"kubernetes.io/projected/eb38e1f8-7da8-4a0e-9416-5f70fbdcac50-kube-api-access-c44qz\") pod \"limitador-limitador-54d6d988d5-c27fh\" (UID: \"eb38e1f8-7da8-4a0e-9416-5f70fbdcac50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" Jul 05 12:03:07.468002 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.467958 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eb38e1f8-7da8-4a0e-9416-5f70fbdcac50-config-file\") pod \"limitador-limitador-54d6d988d5-c27fh\" (UID: \"eb38e1f8-7da8-4a0e-9416-5f70fbdcac50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" Jul 05 12:03:07.568968 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.568934 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c44qz\" (UniqueName: \"kubernetes.io/projected/eb38e1f8-7da8-4a0e-9416-5f70fbdcac50-kube-api-access-c44qz\") pod \"limitador-limitador-54d6d988d5-c27fh\" (UID: \"eb38e1f8-7da8-4a0e-9416-5f70fbdcac50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" Jul 05 12:03:07.568968 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.568972 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eb38e1f8-7da8-4a0e-9416-5f70fbdcac50-config-file\") pod \"limitador-limitador-54d6d988d5-c27fh\" (UID: \"eb38e1f8-7da8-4a0e-9416-5f70fbdcac50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" Jul 05 12:03:07.569578 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.569559 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eb38e1f8-7da8-4a0e-9416-5f70fbdcac50-config-file\") pod \"limitador-limitador-54d6d988d5-c27fh\" (UID: \"eb38e1f8-7da8-4a0e-9416-5f70fbdcac50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" Jul 05 12:03:07.576945 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.576921 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c44qz\" (UniqueName: \"kubernetes.io/projected/eb38e1f8-7da8-4a0e-9416-5f70fbdcac50-kube-api-access-c44qz\") pod \"limitador-limitador-54d6d988d5-c27fh\" (UID: \"eb38e1f8-7da8-4a0e-9416-5f70fbdcac50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" Jul 05 12:03:07.650946 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.650912 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" Jul 05 12:03:07.786027 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.786000 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:03:07.788454 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:03:07.788425 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb38e1f8_7da8_4a0e_9416_5f70fbdcac50.slice/crio-3bf554fbfad9b83fe78a6c72acf2f16696cd0b834ab7df9011feaca504a25418 WatchSource:0}: Error finding container 3bf554fbfad9b83fe78a6c72acf2f16696cd0b834ab7df9011feaca504a25418: Status 404 returned error can't find the container with id 3bf554fbfad9b83fe78a6c72acf2f16696cd0b834ab7df9011feaca504a25418 Jul 05 12:03:07.937153 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.937116 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:07.974635 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:07.974597 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" event={"ID":"eb38e1f8-7da8-4a0e-9416-5f70fbdcac50","Type":"ContainerStarted","Data":"3bf554fbfad9b83fe78a6c72acf2f16696cd0b834ab7df9011feaca504a25418"} Jul 05 12:03:08.012815 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:08.012726 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:10.991544 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:10.991481 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" event={"ID":"eb38e1f8-7da8-4a0e-9416-5f70fbdcac50","Type":"ContainerStarted","Data":"bc50d356d9dd734f13c680c36c7d4ffdc8c01df48391b3f87ac2de4fb079bd72"} Jul 05 12:03:10.991964 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:10.991591 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" Jul 05 12:03:11.008821 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:11.008766 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" podStartSLOduration=1.383263106 podStartE2EDuration="4.00875057s" totalImagesPullingTime="2.625487464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:03:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:03:11.006802444 +0000 UTC m=+591.397504185" watchObservedRunningTime="2026-07-05 12:03:11.00875057 +0000 UTC m=+591.399452331" Jul 05 12:03:20.062041 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:20.062004 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:03:20.066569 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:20.066543 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:03:21.997620 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:21.997584 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-c27fh" Jul 05 12:03:26.939030 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:26.938990 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:33.112429 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:33.112384 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:39.144199 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:39.144165 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:39.670245 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:39.670208 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:46.139214 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:46.139107 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:46.231317 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:46.231280 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:46.289957 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:46.289917 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:46.538256 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:46.538167 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:49.119240 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:49.119192 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:03:49.215800 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:03:49.215767 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:04:21.491263 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:21.491227 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:04:21.640612 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:21.640573 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:04:21.738538 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:21.738506 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:04:21.923298 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:21.923257 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:04:22.356837 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:22.356798 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:04:22.453341 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:22.453296 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:04:22.523607 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:22.523571 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:04:22.588387 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:22.588349 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:04:32.506191 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:32.506150 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:04:32.605662 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:32.605626 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:04:43.126730 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:43.126692 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:04:43.657233 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:04:43.657194 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:05:01.424951 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:01.424907 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:05:02.100791 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:02.100753 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:05:16.402939 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:16.402839 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:05:17.591784 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:17.591747 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:05:22.580865 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.580829 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs"] Jul 05 12:05:22.584531 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.584504 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.588647 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.588620 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 05 12:05:22.588786 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.588668 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 05 12:05:22.588858 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.588785 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 05 12:05:22.588962 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.588854 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7msdg\"" Jul 05 12:05:22.592859 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.592423 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs"] Jul 05 12:05:22.617955 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.617916 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:05:22.649210 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.649171 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4872c6-9576-4c62-9dac-a515bc97c92f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.649391 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.649223 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.649391 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.649256 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zw7m\" (UniqueName: \"kubernetes.io/projected/ec4872c6-9576-4c62-9dac-a515bc97c92f-kube-api-access-8zw7m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.649391 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.649284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.649391 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.649309 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.649391 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.649341 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.750033 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.749996 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.750230 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.750051 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zw7m\" (UniqueName: \"kubernetes.io/projected/ec4872c6-9576-4c62-9dac-a515bc97c92f-kube-api-access-8zw7m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.750230 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.750104 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.750230 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.750143 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.750230 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.750169 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.750230 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.750204 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4872c6-9576-4c62-9dac-a515bc97c92f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.750512 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.750487 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.750559 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.750524 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.750559 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.750545 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.752516 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.752490 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec4872c6-9576-4c62-9dac-a515bc97c92f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.752732 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.752712 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4872c6-9576-4c62-9dac-a515bc97c92f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.757718 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.757693 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zw7m\" (UniqueName: \"kubernetes.io/projected/ec4872c6-9576-4c62-9dac-a515bc97c92f-kube-api-access-8zw7m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs\" (UID: \"ec4872c6-9576-4c62-9dac-a515bc97c92f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:22.896948 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:22.896841 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:23.029800 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:23.029772 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs"] Jul 05 12:05:23.031787 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:05:23.031761 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec4872c6_9576_4c62_9dac_a515bc97c92f.slice/crio-7b566a50a9bf37f45a0d99a9105dde04675039a2d9d79b8dde60eb74ab01dd47 WatchSource:0}: Error finding container 7b566a50a9bf37f45a0d99a9105dde04675039a2d9d79b8dde60eb74ab01dd47: Status 404 returned error can't find the container with id 7b566a50a9bf37f45a0d99a9105dde04675039a2d9d79b8dde60eb74ab01dd47 Jul 05 12:05:23.033450 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:23.033426 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 12:05:23.609742 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:23.609706 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" event={"ID":"ec4872c6-9576-4c62-9dac-a515bc97c92f","Type":"ContainerStarted","Data":"7b566a50a9bf37f45a0d99a9105dde04675039a2d9d79b8dde60eb74ab01dd47"} Jul 05 12:05:24.291693 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:24.291657 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:05:24.615254 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:24.615222 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" event={"ID":"ec4872c6-9576-4c62-9dac-a515bc97c92f","Type":"ContainerStarted","Data":"250f15159764f68c03f8ead09cab8772c0498e46e8b7b6e51d558f926efd6157"} Jul 05 12:05:24.634142 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:24.634062 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" podStartSLOduration=1.597897577 podStartE2EDuration="2.634042209s" totalImagesPullingTime="1.036144632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:05:24.632513226 +0000 UTC m=+725.023214991" watchObservedRunningTime="2026-07-05 12:05:24.634042209 +0000 UTC m=+725.024743964" Jul 05 12:05:32.897984 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:32.897940 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:32.898490 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:32.898011 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:32.910547 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:32.910516 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:33.670505 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:33.670476 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs" Jul 05 12:05:34.911614 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:34.911568 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:05:36.690902 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:36.690848 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:05:45.126192 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:45.126147 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:05:47.387977 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:05:47.387941 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:06:26.614744 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:06:26.614699 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:06:26.715325 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:06:26.715289 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:07:04.488384 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:07:04.488344 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:07:04.588382 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:07:04.588337 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:07:05.671748 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:07:05.671707 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:07:05.770623 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:07:05.770585 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:07:32.188151 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:07:32.188111 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:07:42.795052 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:07:42.795015 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:07:54.389710 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:07:54.389678 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:08:02.881743 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:02.881697 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:08:03.245068 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:03.244984 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:08:13.382753 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:13.382666 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:08:15.132055 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:15.132015 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:08:16.329461 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:16.329426 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:08:17.847244 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:17.847209 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:08:20.099222 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:20.099194 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:08:20.099751 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:20.099731 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:08:21.253959 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:21.253918 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:08:53.455475 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:53.455432 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:08:53.555549 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:53.555515 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:08:53.681882 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:53.681848 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:08:59.333198 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:59.333157 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:08:59.441308 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:59.441268 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:08:59.533279 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:59.533232 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:08:59.688766 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:08:59.688670 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:09:05.296261 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:09:05.296226 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:09:05.396590 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:09:05.396554 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:09:22.091841 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:09:22.091801 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:09:31.686930 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:09:31.686892 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:09:41.686026 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:09:41.685981 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:09:50.990376 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:09:50.990301 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:10:01.986554 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:10:01.986512 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:10:10.693989 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:10:10.693949 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:10:22.982279 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:10:22.982240 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:10:30.086694 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:10:30.086650 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:10:31.528891 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:10:31.528839 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:10:37.389076 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:10:37.389037 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:10:46.784808 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:10:46.784775 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:10:58.483457 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:10:58.483416 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:11:08.981271 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:11:08.981228 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:11:32.987582 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:11:32.987494 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:11:44.828365 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:11:44.828325 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:11:59.296634 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:11:59.296595 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:12:10.486967 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:12:10.486927 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:12:19.495787 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:12:19.495747 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:12:28.391056 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:12:28.391019 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:12:36.998011 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:12:36.997968 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:13:04.385182 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:04.385086 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6"] Jul 05 12:13:04.385731 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:04.385320 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" podUID="84674c0b-aeaa-45df-8e5e-d8f87a080c54" containerName="manager" containerID="cri-o://4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9" gracePeriod=10 Jul 05 12:13:06.453729 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.453701 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:13:06.568935 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.568899 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jnf4\" (UniqueName: \"kubernetes.io/projected/84674c0b-aeaa-45df-8e5e-d8f87a080c54-kube-api-access-4jnf4\") pod \"84674c0b-aeaa-45df-8e5e-d8f87a080c54\" (UID: \"84674c0b-aeaa-45df-8e5e-d8f87a080c54\") " Jul 05 12:13:06.569112 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.568967 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84674c0b-aeaa-45df-8e5e-d8f87a080c54-extensions-socket-volume\") pod \"84674c0b-aeaa-45df-8e5e-d8f87a080c54\" (UID: \"84674c0b-aeaa-45df-8e5e-d8f87a080c54\") " Jul 05 12:13:06.569367 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.569341 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84674c0b-aeaa-45df-8e5e-d8f87a080c54-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "84674c0b-aeaa-45df-8e5e-d8f87a080c54" (UID: "84674c0b-aeaa-45df-8e5e-d8f87a080c54"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:13:06.571048 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.571020 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84674c0b-aeaa-45df-8e5e-d8f87a080c54-kube-api-access-4jnf4" (OuterVolumeSpecName: "kube-api-access-4jnf4") pod "84674c0b-aeaa-45df-8e5e-d8f87a080c54" (UID: "84674c0b-aeaa-45df-8e5e-d8f87a080c54"). InnerVolumeSpecName "kube-api-access-4jnf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:13:06.670079 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.670041 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jnf4\" (UniqueName: \"kubernetes.io/projected/84674c0b-aeaa-45df-8e5e-d8f87a080c54-kube-api-access-4jnf4\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:13:06.670079 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.670075 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84674c0b-aeaa-45df-8e5e-d8f87a080c54-extensions-socket-volume\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:13:06.819130 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.819044 2575 generic.go:356] "Generic (PLEG): container finished" podID="84674c0b-aeaa-45df-8e5e-d8f87a080c54" containerID="4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9" exitCode=0 Jul 05 12:13:06.819130 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.819096 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" Jul 05 12:13:06.819308 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.819099 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" event={"ID":"84674c0b-aeaa-45df-8e5e-d8f87a080c54","Type":"ContainerDied","Data":"4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9"} Jul 05 12:13:06.819308 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.819219 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6" event={"ID":"84674c0b-aeaa-45df-8e5e-d8f87a080c54","Type":"ContainerDied","Data":"996ac4ce058cedef40673e33aaecf7808043b4ef863e77e6f40f05a64e123130"} Jul 05 12:13:06.819308 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.819242 2575 scope.go:117] "RemoveContainer" containerID="4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9" Jul 05 12:13:06.829231 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.829211 2575 scope.go:117] "RemoveContainer" containerID="4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9" Jul 05 12:13:06.829497 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:13:06.829479 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9\": container with ID starting with 4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9 not found: ID does not exist" containerID="4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9" Jul 05 12:13:06.829538 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.829506 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9"} err="failed to get container status \"4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9\": rpc error: code = NotFound desc = could not find container \"4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9\": container with ID starting with 4bcba2bd5c10ee0a939905480120cedb0ee3508ddbe2302ce372cd1c5e65b7d9 not found: ID does not exist" Jul 05 12:13:06.840768 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.840745 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6"] Jul 05 12:13:06.847358 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:06.847286 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4spd6"] Jul 05 12:13:08.129563 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:08.129527 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84674c0b-aeaa-45df-8e5e-d8f87a080c54" path="/var/lib/kubelet/pods/84674c0b-aeaa-45df-8e5e-d8f87a080c54/volumes" Jul 05 12:13:20.129055 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:20.129015 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:13:20.129732 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:20.129712 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:13:48.956815 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:48.956757 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:13:50.141722 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:13:50.141670 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:14:05.565618 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.565582 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx"] Jul 05 12:14:05.567981 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.565998 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84674c0b-aeaa-45df-8e5e-d8f87a080c54" containerName="manager" Jul 05 12:14:05.567981 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.566015 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="84674c0b-aeaa-45df-8e5e-d8f87a080c54" containerName="manager" Jul 05 12:14:05.567981 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.566160 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="84674c0b-aeaa-45df-8e5e-d8f87a080c54" containerName="manager" Jul 05 12:14:05.568979 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.568956 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" Jul 05 12:14:05.574271 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.574249 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r799m\"" Jul 05 12:14:05.584152 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.584130 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx"] Jul 05 12:14:05.662529 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.662486 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgq4d\" (UniqueName: \"kubernetes.io/projected/bda3f19b-66ee-4a36-ab55-5ca30e2e6c46-kube-api-access-xgq4d\") pod \"kuadrant-operator-controller-manager-86878c769f-frmqx\" (UID: \"bda3f19b-66ee-4a36-ab55-5ca30e2e6c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" Jul 05 12:14:05.662720 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.662605 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bda3f19b-66ee-4a36-ab55-5ca30e2e6c46-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-frmqx\" (UID: \"bda3f19b-66ee-4a36-ab55-5ca30e2e6c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" Jul 05 12:14:05.763096 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.763048 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgq4d\" (UniqueName: \"kubernetes.io/projected/bda3f19b-66ee-4a36-ab55-5ca30e2e6c46-kube-api-access-xgq4d\") pod \"kuadrant-operator-controller-manager-86878c769f-frmqx\" (UID: \"bda3f19b-66ee-4a36-ab55-5ca30e2e6c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" Jul 05 12:14:05.763296 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.763132 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bda3f19b-66ee-4a36-ab55-5ca30e2e6c46-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-frmqx\" (UID: \"bda3f19b-66ee-4a36-ab55-5ca30e2e6c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" Jul 05 12:14:05.763566 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.763544 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bda3f19b-66ee-4a36-ab55-5ca30e2e6c46-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-frmqx\" (UID: \"bda3f19b-66ee-4a36-ab55-5ca30e2e6c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" Jul 05 12:14:05.771673 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.771644 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgq4d\" (UniqueName: \"kubernetes.io/projected/bda3f19b-66ee-4a36-ab55-5ca30e2e6c46-kube-api-access-xgq4d\") pod \"kuadrant-operator-controller-manager-86878c769f-frmqx\" (UID: \"bda3f19b-66ee-4a36-ab55-5ca30e2e6c46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" Jul 05 12:14:05.878440 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:05.878345 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" Jul 05 12:14:06.026000 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:06.025959 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx"] Jul 05 12:14:06.030082 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:14:06.030055 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda3f19b_66ee_4a36_ab55_5ca30e2e6c46.slice/crio-57a9b4097f1b961ae51f283b7949a33a8c9bed5ea912cbae4593d8ff9f29fd03 WatchSource:0}: Error finding container 57a9b4097f1b961ae51f283b7949a33a8c9bed5ea912cbae4593d8ff9f29fd03: Status 404 returned error can't find the container with id 57a9b4097f1b961ae51f283b7949a33a8c9bed5ea912cbae4593d8ff9f29fd03 Jul 05 12:14:06.093945 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:06.093908 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" event={"ID":"bda3f19b-66ee-4a36-ab55-5ca30e2e6c46","Type":"ContainerStarted","Data":"57a9b4097f1b961ae51f283b7949a33a8c9bed5ea912cbae4593d8ff9f29fd03"} Jul 05 12:14:07.103059 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:07.103018 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" event={"ID":"bda3f19b-66ee-4a36-ab55-5ca30e2e6c46","Type":"ContainerStarted","Data":"655728d6b54e078c87ec8dd073967b176042b5d925bbd7b73d729abd69eaa4a1"} Jul 05 12:14:07.103438 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:07.103133 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" Jul 05 12:14:07.123114 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:07.123064 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" podStartSLOduration=2.123047564 podStartE2EDuration="2.123047564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:14:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:14:07.121606385 +0000 UTC m=+1247.512308146" watchObservedRunningTime="2026-07-05 12:14:07.123047564 +0000 UTC m=+1247.513749327" Jul 05 12:14:18.110689 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:18.110660 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-frmqx" Jul 05 12:14:27.087574 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:27.087537 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:14:35.687204 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:35.687168 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:14:48.496273 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:48.496233 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:14:57.284478 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:14:57.284446 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:15:05.986654 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:15:05.986614 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:15:15.893862 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:15:15.893830 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:15:28.787106 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:15:28.787074 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:15:37.285645 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:15:37.285613 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:15:47.194504 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:15:47.194422 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:15:57.184013 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:15:57.183982 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:16:08.494851 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:16:08.494801 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:16:14.692075 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:16:14.692032 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:16:32.079346 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:16:32.079312 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:16:33.810128 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:16:33.810094 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:16:36.892062 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:16:36.892020 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:16:45.691312 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:16:45.691274 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:16:56.185335 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:16:56.185297 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:17:08.291743 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:17:08.291701 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:17:23.785623 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:17:23.785539 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:17:32.488051 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:17:32.488013 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:17:41.486976 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:17:41.486940 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:17:51.688726 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:17:51.688693 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:18:00.691926 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:18:00.691867 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:18:18.891957 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:18:18.891922 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:18:20.158290 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:18:20.158260 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:18:20.159589 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:18:20.159568 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:18:27.597955 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:18:27.597917 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:18:37.594452 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:18:37.594411 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:18:48.188392 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:18:48.188267 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:18:59.686555 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:18:59.686514 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:19:08.388057 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:19:08.388022 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:19:16.001194 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:19:16.001154 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:19:32.685659 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:19:32.685620 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:19:41.785747 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:19:41.785708 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:19:51.187187 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:19:51.187146 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:19:59.186142 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:19:59.186106 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:20:04.250675 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:04.250640 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:05.521159 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:05.521121 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:09.332596 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:09.332563 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:09.438739 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:09.438696 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:09.533145 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:09.533117 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:09.584598 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:09.584523 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:10.725741 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:10.725706 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:10.799296 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:10.799260 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:10.858943 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:10.858906 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:11.219614 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:11.219580 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:11.405756 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:11.405723 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:11.508051 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:11.507963 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:13.150831 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:13.150783 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:13.183999 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:13.183967 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:20:15.921187 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:15.921152 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:16.089701 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:16.089663 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:16.198072 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:16.197986 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:16.298698 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:16.298664 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:19.407348 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:19.407310 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:21.093341 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:21.093308 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:20:21.751374 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:21.751343 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:21.856775 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:21.856742 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:21.951868 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:21.951836 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:22.009460 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:22.009372 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:23.156038 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:23.156006 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:23.264166 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:23.264129 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:23.386114 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:23.386081 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:23.484768 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:23.484682 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:23.796826 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:23.796727 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:23.901758 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:23.901712 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:25.161146 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:25.161109 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:28.336107 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:28.336071 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:28.690805 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:28.690717 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:34.724987 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:34.724947 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:34.831019 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:34.830982 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:40.447140 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:40.447103 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:40.549794 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:40.549757 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:40.646250 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:40.646218 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:41.855282 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:41.855250 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:41.952786 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:41.952730 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:42.111253 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:42.111164 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:42.233723 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:42.233688 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:42.335237 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:42.335201 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:42.434558 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:42.434468 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:42.534628 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:42.534580 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:42.639838 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:42.639806 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:44.493776 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:44.493736 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:20:45.601766 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:45.601714 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:46.402269 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:46.402235 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:46.631709 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:46.631674 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:46.749744 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:46.749660 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:46.858805 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:46.858774 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:48.392394 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:48.392328 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:52.340020 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:52.339977 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:52.406050 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:52.406017 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:52.815329 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:52.815294 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:52.897370 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:52.897328 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:20:52.904304 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:52.904268 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:52.962405 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:52.962364 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:54.423032 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:54.422995 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:20:59.959390 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:20:59.959351 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:00.016177 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:00.016143 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:00.099798 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:00.099753 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:00.201310 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:00.201259 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:00.297342 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:00.297242 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:00.703897 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:00.703855 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:26.717656 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:26.717621 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:32.219360 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:32.219323 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:32.275025 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:32.274991 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:32.346368 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:32.346321 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:32.445419 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:32.445389 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:48.302175 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:48.302085 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:53.897028 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:53.896989 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:53.955663 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:53.955627 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:54.056183 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:54.056136 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:21:54.186549 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:21:54.186455 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:04.660538 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:04.660486 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:05.493884 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:05.493836 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:06.043976 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:06.043937 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:06.144288 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:06.144255 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:11.126889 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:11.126841 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:11.231680 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:11.231635 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:11.327093 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:11.327058 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:11.441675 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:11.441582 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:12.229785 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:12.229748 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:12.325617 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:12.325565 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:12.386949 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:12.386917 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:14.113713 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:14.113670 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:14.492159 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:14.492070 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:22:19.981590 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:19.981559 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:20.048406 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:20.048351 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:20.663305 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:20.663270 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:23.582757 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:23.582719 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:22:26.308301 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:26.308265 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:26.365260 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:26.365207 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:26.548428 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:26.548399 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:26.896028 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:26.895991 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:27.483041 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:27.483003 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:29.806370 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:29.806339 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:29.889258 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:29.889225 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:29.950535 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:29.950497 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:30.276102 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:30.276015 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:30.377285 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:30.377250 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:30.477180 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:30.477141 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:34.324245 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:34.324210 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:35.003894 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:35.003850 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:22:35.005463 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:35.005432 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:35.105542 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:35.105508 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:35.451818 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:35.451785 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:41.058212 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:41.058181 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:41.201324 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:41.201272 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:41.496915 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:41.496786 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:41.891148 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:41.891088 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:41.943386 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:41.943353 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:45.891380 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:45.891341 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:22:47.511396 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:47.511347 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:47.611332 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:47.611295 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:47.782808 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:47.782697 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:47.998510 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:47.998470 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:48.136277 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:48.136247 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:48.235158 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:48.235120 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:48.362790 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:48.362755 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:54.080015 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:54.079974 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:54.138113 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:54.138077 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:54.215053 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:54.215020 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:54.313372 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:54.313341 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:22:54.464103 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:22:54.464017 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:00.041215 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:00.041171 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:00.097194 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:00.097155 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:06.537814 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:06.537774 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:06.872331 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:06.872289 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:07.089170 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:07.089134 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:09.857073 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:09.857037 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:09.926227 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:09.926185 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:09.987982 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:09.987946 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:10.346897 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:10.346849 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:10.521237 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:10.521203 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:10.625502 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:10.625417 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:14.386865 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:14.386768 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:14.997425 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:14.997388 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:23:20.187416 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:20.187381 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:23:20.188713 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:20.188689 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:23:21.385260 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.385224 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn"] Jul 05 12:23:21.389258 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.389234 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.393841 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.393653 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c31da5f5\"/\"e2e-lifecycle-model-c31da5f5-kserve-self-signed-certs\"" Jul 05 12:23:21.393841 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.393681 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c31da5f5\"/\"kube-root-ca.crt\"" Jul 05 12:23:21.394067 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.393894 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c31da5f5\"/\"default-dockercfg-c4f7l\"" Jul 05 12:23:21.394067 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.393913 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c31da5f5\"/\"openshift-service-ca.crt\"" Jul 05 12:23:21.396263 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.396176 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn"] Jul 05 12:23:21.413230 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.413198 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:21.494614 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.494578 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-dshm\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.494614 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.494616 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-home\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.494840 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.494632 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-model-cache\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.494840 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.494729 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-tmp-dir\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.494840 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.494799 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcpqq\" (UniqueName: \"kubernetes.io/projected/629b6bab-3c87-40fa-9e62-64976788bab8-kube-api-access-vcpqq\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.494840 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.494834 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629b6bab-3c87-40fa-9e62-64976788bab8-tls-certs\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.595475 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.595441 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-home\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.595475 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.595475 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-model-cache\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.595700 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.595508 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-tmp-dir\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.595700 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.595537 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcpqq\" (UniqueName: \"kubernetes.io/projected/629b6bab-3c87-40fa-9e62-64976788bab8-kube-api-access-vcpqq\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.595700 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.595555 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629b6bab-3c87-40fa-9e62-64976788bab8-tls-certs\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.595700 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.595604 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-dshm\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.595942 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.595915 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-home\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.596000 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.595921 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-model-cache\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.596000 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.595971 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-tmp-dir\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.597829 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.597802 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-dshm\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.598205 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.598185 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629b6bab-3c87-40fa-9e62-64976788bab8-tls-certs\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.603903 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.603859 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcpqq\" (UniqueName: \"kubernetes.io/projected/629b6bab-3c87-40fa-9e62-64976788bab8-kube-api-access-vcpqq\") pod \"e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.699046 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.698955 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:21.834552 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:21.834406 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn"] Jul 05 12:23:21.837593 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:23:21.837559 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod629b6bab_3c87_40fa_9e62_64976788bab8.slice/crio-ea841ec4f797d3e79966fe2822b099dc4a8e1cbc063539228d018b31b54e9da6 WatchSource:0}: Error finding container ea841ec4f797d3e79966fe2822b099dc4a8e1cbc063539228d018b31b54e9da6: Status 404 returned error can't find the container with id ea841ec4f797d3e79966fe2822b099dc4a8e1cbc063539228d018b31b54e9da6 Jul 05 12:23:22.813555 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:22.813512 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" event={"ID":"629b6bab-3c87-40fa-9e62-64976788bab8","Type":"ContainerStarted","Data":"a41828f620dd00404272f254085c965d83a21d8001e33fde73bf4df729923493"} Jul 05 12:23:22.813555 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:22.813560 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" event={"ID":"629b6bab-3c87-40fa-9e62-64976788bab8","Type":"ContainerStarted","Data":"ea841ec4f797d3e79966fe2822b099dc4a8e1cbc063539228d018b31b54e9da6"} Jul 05 12:23:22.829617 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:22.829567 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" podStartSLOduration=1.829553078 podStartE2EDuration="1.829553078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:23:22.828818805 +0000 UTC m=+1803.219520577" watchObservedRunningTime="2026-07-05 12:23:22.829553078 +0000 UTC m=+1803.220254899" Jul 05 12:23:26.089547 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:26.089511 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:23:31.699578 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:31.699535 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:31.699578 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:31.699583 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:31.712510 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:31.712484 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:31.873177 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:31.873143 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:23:38.894089 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:38.894049 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:23:40.912279 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:40.912244 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:41.013442 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:41.013411 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:41.709745 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:41.709708 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:42.228924 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:42.228867 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:42.329713 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:42.329679 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:42.545157 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:42.545069 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:47.822410 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:47.822361 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:23:52.809172 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:52.809128 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn"] Jul 05 12:23:52.880786 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:52.880749 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:23:52.883374 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:23:52.883341 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:07.829591 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:07.829527 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" podUID="629b6bab-3c87-40fa-9e62-64976788bab8" containerName="main" containerID="cri-o://a41828f620dd00404272f254085c965d83a21d8001e33fde73bf4df729923493" gracePeriod=45 Jul 05 12:24:08.037611 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.037572 2575 generic.go:356] "Generic (PLEG): container finished" podID="629b6bab-3c87-40fa-9e62-64976788bab8" containerID="a41828f620dd00404272f254085c965d83a21d8001e33fde73bf4df729923493" exitCode=0 Jul 05 12:24:08.037768 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.037637 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" event={"ID":"629b6bab-3c87-40fa-9e62-64976788bab8","Type":"ContainerDied","Data":"a41828f620dd00404272f254085c965d83a21d8001e33fde73bf4df729923493"} Jul 05 12:24:08.088592 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.088534 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:24:08.191201 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.191170 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcpqq\" (UniqueName: \"kubernetes.io/projected/629b6bab-3c87-40fa-9e62-64976788bab8-kube-api-access-vcpqq\") pod \"629b6bab-3c87-40fa-9e62-64976788bab8\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " Jul 05 12:24:08.191201 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.191211 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-model-cache\") pod \"629b6bab-3c87-40fa-9e62-64976788bab8\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " Jul 05 12:24:08.191465 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.191262 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-tmp-dir\") pod \"629b6bab-3c87-40fa-9e62-64976788bab8\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " Jul 05 12:24:08.191465 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.191323 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629b6bab-3c87-40fa-9e62-64976788bab8-tls-certs\") pod \"629b6bab-3c87-40fa-9e62-64976788bab8\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " Jul 05 12:24:08.191465 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.191344 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-dshm\") pod \"629b6bab-3c87-40fa-9e62-64976788bab8\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " Jul 05 12:24:08.191465 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.191396 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-home\") pod \"629b6bab-3c87-40fa-9e62-64976788bab8\" (UID: \"629b6bab-3c87-40fa-9e62-64976788bab8\") " Jul 05 12:24:08.191674 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.191569 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-model-cache" (OuterVolumeSpecName: "model-cache") pod "629b6bab-3c87-40fa-9e62-64976788bab8" (UID: "629b6bab-3c87-40fa-9e62-64976788bab8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:24:08.191674 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.191584 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "629b6bab-3c87-40fa-9e62-64976788bab8" (UID: "629b6bab-3c87-40fa-9e62-64976788bab8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:24:08.191744 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.191698 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-home" (OuterVolumeSpecName: "home") pod "629b6bab-3c87-40fa-9e62-64976788bab8" (UID: "629b6bab-3c87-40fa-9e62-64976788bab8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:24:08.193599 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.193571 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/629b6bab-3c87-40fa-9e62-64976788bab8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "629b6bab-3c87-40fa-9e62-64976788bab8" (UID: "629b6bab-3c87-40fa-9e62-64976788bab8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:24:08.193721 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.193602 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-dshm" (OuterVolumeSpecName: "dshm") pod "629b6bab-3c87-40fa-9e62-64976788bab8" (UID: "629b6bab-3c87-40fa-9e62-64976788bab8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:24:08.193721 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.193622 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/629b6bab-3c87-40fa-9e62-64976788bab8-kube-api-access-vcpqq" (OuterVolumeSpecName: "kube-api-access-vcpqq") pod "629b6bab-3c87-40fa-9e62-64976788bab8" (UID: "629b6bab-3c87-40fa-9e62-64976788bab8"). InnerVolumeSpecName "kube-api-access-vcpqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:24:08.293130 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.293088 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-home\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:24:08.293130 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.293138 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcpqq\" (UniqueName: \"kubernetes.io/projected/629b6bab-3c87-40fa-9e62-64976788bab8-kube-api-access-vcpqq\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:24:08.293350 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.293152 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-model-cache\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:24:08.293350 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.293167 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-tmp-dir\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:24:08.293350 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.293178 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629b6bab-3c87-40fa-9e62-64976788bab8-tls-certs\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:24:08.293350 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:08.293190 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629b6bab-3c87-40fa-9e62-64976788bab8-dshm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:24:09.045081 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:09.045047 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" Jul 05 12:24:09.047744 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:09.045050 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn" event={"ID":"629b6bab-3c87-40fa-9e62-64976788bab8","Type":"ContainerDied","Data":"ea841ec4f797d3e79966fe2822b099dc4a8e1cbc063539228d018b31b54e9da6"} Jul 05 12:24:09.047744 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:09.045175 2575 scope.go:117] "RemoveContainer" containerID="a41828f620dd00404272f254085c965d83a21d8001e33fde73bf4df729923493" Jul 05 12:24:09.068551 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:09.068516 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn"] Jul 05 12:24:09.078886 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:09.078846 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c31da5f5/e2e-lifecycle-model-c31da5f5-kserve-567c4c676c-phknn"] Jul 05 12:24:10.129524 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:10.129484 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="629b6bab-3c87-40fa-9e62-64976788bab8" path="/var/lib/kubelet/pods/629b6bab-3c87-40fa-9e62-64976788bab8/volumes" Jul 05 12:24:28.940147 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:28.940096 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:29.761659 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:29.761627 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:32.290651 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:32.290616 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:24:35.161813 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:35.161775 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:35.261585 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:35.261547 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:35.433258 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:35.433164 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:35.677853 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:35.677819 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:36.747408 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:36.747379 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:36.845568 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:36.845535 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:36.945510 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:36.945480 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:37.724212 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:37.724174 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:37.823092 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:37.823057 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:37.923296 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:37.923264 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:39.387183 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:39.387143 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:45.153650 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:45.153573 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:45.253637 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:45.253598 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:45.845834 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:45.845791 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:51.474768 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:51.474732 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:51.524198 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:51.524166 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:51.621416 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:51.621381 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:24:51.722081 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:24:51.722046 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:02.229517 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:02.229479 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:02.591473 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:02.591442 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:23.574213 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:23.574175 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:29.345332 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:29.345302 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:29.451650 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:29.451584 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:29.742399 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:29.742316 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:29.994339 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:29.994231 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:30.054386 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:30.054352 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:30.164759 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:30.164725 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:32.715651 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:32.715621 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:32.814783 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:32.814751 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:32.923315 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:32.923282 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:33.467277 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:33.467247 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:33.568091 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:33.568052 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:33.675264 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:33.675186 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:36.985641 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:36.985600 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:25:37.279031 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:37.278942 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:37.604480 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:37.604439 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:37.938900 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:37.938790 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:38.200640 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:38.200545 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:40.721179 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:40.721144 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:40.823181 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:40.823148 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:40.879044 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:40.879005 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:41.288862 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:41.288830 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:41.403123 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:41.403092 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:41.503767 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:41.503730 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:45.241258 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:45.241226 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:48.386963 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:48.386928 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:25:54.899312 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:54.899272 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:55.000656 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:55.000618 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:58.995026 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:58.994989 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp"] Jul 05 12:25:58.995470 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:58.995449 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="629b6bab-3c87-40fa-9e62-64976788bab8" containerName="main" Jul 05 12:25:58.995470 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:58.995471 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="629b6bab-3c87-40fa-9e62-64976788bab8" containerName="main" Jul 05 12:25:58.995566 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:58.995542 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="629b6bab-3c87-40fa-9e62-64976788bab8" containerName="main" Jul 05 12:25:58.998732 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:58.998714 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.003079 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.003052 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1d2467\"/\"default-dockercfg-jtlvg\"" Jul 05 12:25:59.003227 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.003084 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1d2467\"/\"kube-root-ca.crt\"" Jul 05 12:25:59.003227 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.003117 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1d2467\"/\"openshift-service-ca.crt\"" Jul 05 12:25:59.003227 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.003197 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1d2467\"/\"test-model-1d2467-kserve-self-signed-certs\"" Jul 05 12:25:59.006560 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.006537 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp"] Jul 05 12:25:59.023606 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.023572 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:25:59.140414 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.140372 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tmp-dir\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.140638 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.140420 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-dshm\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.140638 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.140450 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tls-certs\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.140638 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.140512 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-home\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.140638 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.140589 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-model-cache\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.140638 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.140613 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9dzm\" (UniqueName: \"kubernetes.io/projected/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-kube-api-access-x9dzm\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.241927 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.241885 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-home\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.242135 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.241980 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-model-cache\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.242135 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.242002 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9dzm\" (UniqueName: \"kubernetes.io/projected/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-kube-api-access-x9dzm\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.242251 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.242134 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tmp-dir\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.242251 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.242161 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-dshm\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.242251 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.242185 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tls-certs\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.242404 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.242349 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-home\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.242463 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.242395 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-model-cache\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.242678 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.242654 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tmp-dir\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.244507 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.244485 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-dshm\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.244807 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.244789 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tls-certs\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.251005 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.250938 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9dzm\" (UniqueName: \"kubernetes.io/projected/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-kube-api-access-x9dzm\") pod \"test-model-1d2467-kserve-7ccdcbbb97-bzhgp\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.308077 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.308029 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:25:59.451657 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.451627 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp"] Jul 05 12:25:59.453297 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:25:59.453270 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4e9c75b_1d2a_4230_8613_6b436dc8aa0a.slice/crio-f5d1a6c5fdcffdcbe592e41bfb1af02a878a3c7c1c09504cba8bf456ce6d1e58 WatchSource:0}: Error finding container f5d1a6c5fdcffdcbe592e41bfb1af02a878a3c7c1c09504cba8bf456ce6d1e58: Status 404 returned error can't find the container with id f5d1a6c5fdcffdcbe592e41bfb1af02a878a3c7c1c09504cba8bf456ce6d1e58 Jul 05 12:25:59.569791 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.569758 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" event={"ID":"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a","Type":"ContainerStarted","Data":"f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4"} Jul 05 12:25:59.569975 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.569803 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" event={"ID":"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a","Type":"ContainerStarted","Data":"f5d1a6c5fdcffdcbe592e41bfb1af02a878a3c7c1c09504cba8bf456ce6d1e58"} Jul 05 12:25:59.587896 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:25:59.587821 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" podStartSLOduration=1.587805741 podStartE2EDuration="1.587805741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:25:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:25:59.585369684 +0000 UTC m=+1959.976071457" watchObservedRunningTime="2026-07-05 12:25:59.587805741 +0000 UTC m=+1959.978507503" Jul 05 12:26:02.288993 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:02.288957 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:26:02.326341 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:02.326310 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:02.376983 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:02.376952 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:02.937040 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:02.936992 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:03.037998 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:03.037962 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:09.309021 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:09.308983 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:26:09.309478 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:09.309176 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:26:09.321899 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:09.321846 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:26:09.629123 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:09.629043 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:26:12.783916 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:12.783805 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:26:25.893665 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.893623 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv"] Jul 05 12:26:25.897269 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.897252 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:25.901418 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.901393 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-40cd04\"/\"default-dockercfg-kczc9\"" Jul 05 12:26:25.901549 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.901448 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-40cd04\"/\"kube-root-ca.crt\"" Jul 05 12:26:25.901549 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.901393 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-40cd04\"/\"openshift-service-ca.crt\"" Jul 05 12:26:25.901549 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.901509 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-40cd04\"/\"test-model-40cd04-kserve-self-signed-certs\"" Jul 05 12:26:25.905353 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.905324 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv"] Jul 05 12:26:25.921797 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.921764 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:25.980416 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.980382 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-home\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:25.980586 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.980433 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f35350a7-092e-4be6-8264-c73556c4466a-tls-certs\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:25.980586 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.980461 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-tmp-dir\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:25.980586 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.980479 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-dshm\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:25.980586 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.980501 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-model-cache\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:25.980586 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:25.980521 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbc4d\" (UniqueName: \"kubernetes.io/projected/f35350a7-092e-4be6-8264-c73556c4466a-kube-api-access-pbc4d\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.081440 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.081391 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f35350a7-092e-4be6-8264-c73556c4466a-tls-certs\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.081440 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.081444 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-tmp-dir\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.081715 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.081465 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-dshm\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.081715 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.081658 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-model-cache\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.081715 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.081703 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbc4d\" (UniqueName: \"kubernetes.io/projected/f35350a7-092e-4be6-8264-c73556c4466a-kube-api-access-pbc4d\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.081921 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.081827 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-home\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.081921 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.081908 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-tmp-dir\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.082056 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.082038 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-model-cache\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.082211 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.082189 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-home\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.083808 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.083776 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-dshm\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.083987 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.083971 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f35350a7-092e-4be6-8264-c73556c4466a-tls-certs\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.091906 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.091861 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbc4d\" (UniqueName: \"kubernetes.io/projected/f35350a7-092e-4be6-8264-c73556c4466a-kube-api-access-pbc4d\") pod \"test-model-40cd04-kserve-649b5f7c87-s2hlv\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.207085 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.207005 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:26.348920 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.348897 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv"] Jul 05 12:26:26.351207 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:26:26.351178 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf35350a7_092e_4be6_8264_c73556c4466a.slice/crio-ab794e589c91f73f11c3f8684962ab07733dcb6712bec75019c4bac5281b68f6 WatchSource:0}: Error finding container ab794e589c91f73f11c3f8684962ab07733dcb6712bec75019c4bac5281b68f6: Status 404 returned error can't find the container with id ab794e589c91f73f11c3f8684962ab07733dcb6712bec75019c4bac5281b68f6 Jul 05 12:26:26.704153 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.704120 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" event={"ID":"f35350a7-092e-4be6-8264-c73556c4466a","Type":"ContainerStarted","Data":"809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3"} Jul 05 12:26:26.704153 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.704160 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" event={"ID":"f35350a7-092e-4be6-8264-c73556c4466a","Type":"ContainerStarted","Data":"ab794e589c91f73f11c3f8684962ab07733dcb6712bec75019c4bac5281b68f6"} Jul 05 12:26:26.723493 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:26.723446 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" podStartSLOduration=1.723432062 podStartE2EDuration="1.723432062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:26:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:26:26.721788657 +0000 UTC m=+1987.112490421" watchObservedRunningTime="2026-07-05 12:26:26.723432062 +0000 UTC m=+1987.114133865" Jul 05 12:26:31.303142 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:31.303106 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:26:36.207443 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:36.207406 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:36.207902 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:36.207456 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:36.220345 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:36.220315 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:36.766626 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:36.766595 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:26:48.893910 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:48.893857 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp"] Jul 05 12:26:48.918075 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:48.918039 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:49.853155 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:49.853119 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv"] Jul 05 12:26:49.871917 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:49.871862 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:50.037953 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:50.037901 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:50.458601 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:50.458566 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:50.557089 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:50.557059 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:50.658566 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:50.658533 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:50.743359 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:50.743266 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:26:53.791789 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:53.791752 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:26:55.538528 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:26:55.538490 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:03.914293 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:03.914229 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" podUID="b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" containerName="main" containerID="cri-o://f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4" gracePeriod=45 Jul 05 12:27:04.184280 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.184253 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:27:04.323427 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.323398 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tls-certs\") pod \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " Jul 05 12:27:04.323632 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.323468 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9dzm\" (UniqueName: \"kubernetes.io/projected/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-kube-api-access-x9dzm\") pod \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " Jul 05 12:27:04.323632 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.323541 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-model-cache\") pod \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " Jul 05 12:27:04.323632 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.323564 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tmp-dir\") pod \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " Jul 05 12:27:04.323632 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.323600 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-home\") pod \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " Jul 05 12:27:04.323898 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.323641 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-dshm\") pod \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\" (UID: \"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a\") " Jul 05 12:27:04.323898 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.323818 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" (UID: "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:27:04.323898 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.323854 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-model-cache" (OuterVolumeSpecName: "model-cache") pod "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" (UID: "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:27:04.324047 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.323867 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-home" (OuterVolumeSpecName: "home") pod "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" (UID: "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:27:04.325843 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.325814 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-dshm" (OuterVolumeSpecName: "dshm") pod "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" (UID: "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:27:04.325973 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.325814 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-kube-api-access-x9dzm" (OuterVolumeSpecName: "kube-api-access-x9dzm") pod "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" (UID: "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a"). InnerVolumeSpecName "kube-api-access-x9dzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:27:04.325973 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.325859 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" (UID: "b4e9c75b-1d2a-4230-8613-6b436dc8aa0a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:27:04.424863 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.424770 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-dshm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:04.424863 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.424804 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tls-certs\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:04.424863 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.424816 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9dzm\" (UniqueName: \"kubernetes.io/projected/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-kube-api-access-x9dzm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:04.424863 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.424825 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-model-cache\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:04.424863 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.424854 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-tmp-dir\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:04.424863 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.424862 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a-home\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:04.874132 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.874085 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" podUID="f35350a7-092e-4be6-8264-c73556c4466a" containerName="main" containerID="cri-o://809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3" gracePeriod=45 Jul 05 12:27:04.904516 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.904475 2575 generic.go:356] "Generic (PLEG): container finished" podID="b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" containerID="f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4" exitCode=0 Jul 05 12:27:04.904689 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.904533 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" Jul 05 12:27:04.904689 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.904555 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" event={"ID":"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a","Type":"ContainerDied","Data":"f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4"} Jul 05 12:27:04.904689 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.904613 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp" event={"ID":"b4e9c75b-1d2a-4230-8613-6b436dc8aa0a","Type":"ContainerDied","Data":"f5d1a6c5fdcffdcbe592e41bfb1af02a878a3c7c1c09504cba8bf456ce6d1e58"} Jul 05 12:27:04.904689 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.904636 2575 scope.go:117] "RemoveContainer" containerID="f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4" Jul 05 12:27:04.938085 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.938052 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp"] Jul 05 12:27:04.944583 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.944550 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-1d2467/test-model-1d2467-kserve-7ccdcbbb97-bzhgp"] Jul 05 12:27:04.985542 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.985419 2575 scope.go:117] "RemoveContainer" containerID="f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4" Jul 05 12:27:04.985779 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:27:04.985756 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4\": container with ID starting with f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4 not found: ID does not exist" containerID="f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4" Jul 05 12:27:04.985829 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:04.985794 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4"} err="failed to get container status \"f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4\": rpc error: code = NotFound desc = could not find container \"f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4\": container with ID starting with f78da6891af8e54b4f52379ed7f112dc8c42c7d47d9b709212954c520fbc4ef4 not found: ID does not exist" Jul 05 12:27:05.133571 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.133511 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:27:05.233216 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.233184 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-tmp-dir\") pod \"f35350a7-092e-4be6-8264-c73556c4466a\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " Jul 05 12:27:05.233216 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.233228 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f35350a7-092e-4be6-8264-c73556c4466a-tls-certs\") pod \"f35350a7-092e-4be6-8264-c73556c4466a\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " Jul 05 12:27:05.233487 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.233246 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-model-cache\") pod \"f35350a7-092e-4be6-8264-c73556c4466a\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " Jul 05 12:27:05.233487 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.233275 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-dshm\") pod \"f35350a7-092e-4be6-8264-c73556c4466a\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " Jul 05 12:27:05.233487 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.233305 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbc4d\" (UniqueName: \"kubernetes.io/projected/f35350a7-092e-4be6-8264-c73556c4466a-kube-api-access-pbc4d\") pod \"f35350a7-092e-4be6-8264-c73556c4466a\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " Jul 05 12:27:05.233487 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.233343 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-home\") pod \"f35350a7-092e-4be6-8264-c73556c4466a\" (UID: \"f35350a7-092e-4be6-8264-c73556c4466a\") " Jul 05 12:27:05.233709 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.233505 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-model-cache" (OuterVolumeSpecName: "model-cache") pod "f35350a7-092e-4be6-8264-c73556c4466a" (UID: "f35350a7-092e-4be6-8264-c73556c4466a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:27:05.233709 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.233521 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f35350a7-092e-4be6-8264-c73556c4466a" (UID: "f35350a7-092e-4be6-8264-c73556c4466a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:27:05.233709 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.233680 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-home" (OuterVolumeSpecName: "home") pod "f35350a7-092e-4be6-8264-c73556c4466a" (UID: "f35350a7-092e-4be6-8264-c73556c4466a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:27:05.233709 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.233703 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-tmp-dir\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:05.233845 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.233724 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-model-cache\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:05.235504 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.235481 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-dshm" (OuterVolumeSpecName: "dshm") pod "f35350a7-092e-4be6-8264-c73556c4466a" (UID: "f35350a7-092e-4be6-8264-c73556c4466a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:27:05.235580 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.235547 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f35350a7-092e-4be6-8264-c73556c4466a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f35350a7-092e-4be6-8264-c73556c4466a" (UID: "f35350a7-092e-4be6-8264-c73556c4466a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:27:05.235727 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.235704 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35350a7-092e-4be6-8264-c73556c4466a-kube-api-access-pbc4d" (OuterVolumeSpecName: "kube-api-access-pbc4d") pod "f35350a7-092e-4be6-8264-c73556c4466a" (UID: "f35350a7-092e-4be6-8264-c73556c4466a"). InnerVolumeSpecName "kube-api-access-pbc4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:27:05.335350 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.335304 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-home\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:05.335350 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.335347 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f35350a7-092e-4be6-8264-c73556c4466a-tls-certs\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:05.335564 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.335363 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f35350a7-092e-4be6-8264-c73556c4466a-dshm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:05.335564 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.335378 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbc4d\" (UniqueName: \"kubernetes.io/projected/f35350a7-092e-4be6-8264-c73556c4466a-kube-api-access-pbc4d\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:27:05.912050 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.912005 2575 generic.go:356] "Generic (PLEG): container finished" podID="f35350a7-092e-4be6-8264-c73556c4466a" containerID="809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3" exitCode=0 Jul 05 12:27:05.912256 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.912064 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" Jul 05 12:27:05.912256 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.912080 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" event={"ID":"f35350a7-092e-4be6-8264-c73556c4466a","Type":"ContainerDied","Data":"809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3"} Jul 05 12:27:05.912256 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.912123 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv" event={"ID":"f35350a7-092e-4be6-8264-c73556c4466a","Type":"ContainerDied","Data":"ab794e589c91f73f11c3f8684962ab07733dcb6712bec75019c4bac5281b68f6"} Jul 05 12:27:05.912256 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.912140 2575 scope.go:117] "RemoveContainer" containerID="809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3" Jul 05 12:27:05.921617 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.921597 2575 scope.go:117] "RemoveContainer" containerID="809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3" Jul 05 12:27:05.921862 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:27:05.921844 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3\": container with ID starting with 809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3 not found: ID does not exist" containerID="809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3" Jul 05 12:27:05.921966 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.921896 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3"} err="failed to get container status \"809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3\": rpc error: code = NotFound desc = could not find container \"809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3\": container with ID starting with 809a3aa95c41a4ca34ffdd3c27f348aa6d73035b06afdd5c4402c8df80ed45f3 not found: ID does not exist" Jul 05 12:27:05.935054 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.935025 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv"] Jul 05 12:27:05.939513 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:05.939488 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-40cd04/test-model-40cd04-kserve-649b5f7c87-s2hlv"] Jul 05 12:27:06.129083 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:06.129051 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" path="/var/lib/kubelet/pods/b4e9c75b-1d2a-4230-8613-6b436dc8aa0a/volumes" Jul 05 12:27:06.129457 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:06.129444 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f35350a7-092e-4be6-8264-c73556c4466a" path="/var/lib/kubelet/pods/f35350a7-092e-4be6-8264-c73556c4466a/volumes" Jul 05 12:27:32.547448 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:32.547409 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:32.821209 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:32.821176 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:33.069261 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:33.069226 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:33.409466 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:33.409424 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:33.851409 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:33.851350 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:34.102746 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:34.102645 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:34.223951 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:34.223906 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:34.358666 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:34.358574 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:36.902099 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:36.902062 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:27:39.927285 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:39.927248 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:39.987927 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:39.987895 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:40.218946 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:40.218823 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:40.448151 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:40.448114 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:40.806616 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:40.806584 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:41.133031 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:41.132952 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:41.233935 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:41.233901 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:44.481813 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:44.481725 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:44.588818 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:44.588770 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:44.679018 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:44.678986 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:44.948698 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:44.948660 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:45.135557 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:45.135523 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:45.235975 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:45.235847 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:48.992262 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:48.992225 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:49.485365 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:49.485330 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:49.825481 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:49.825450 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:50.078084 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:50.077983 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:50.394413 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:50.394335 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:27:52.501917 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:52.501836 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:52.587777 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:52.587743 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:52.649523 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:52.649489 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:53.081338 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:53.081307 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:53.181540 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:53.181507 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:27:57.005334 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:27:57.005298 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:02.692570 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:02.692528 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:28:05.236959 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:05.236917 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:05.641758 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:05.641721 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:05.745918 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:05.745858 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:05.846535 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:05.846498 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:05.947697 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:05.947583 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:11.289117 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:11.289063 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:11.349047 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:11.348988 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:11.594831 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:11.594788 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:11.777688 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:11.777654 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:12.076710 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:12.076677 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:12.450587 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:12.450496 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:12.551199 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:12.551161 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:12.650940 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:12.650909 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:12.751232 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:12.751155 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:16.190158 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:16.190118 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:28:18.166373 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:18.166341 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:18.266993 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:18.266954 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:18.434856 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:18.434760 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:18.666363 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:18.666327 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:19.013899 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:19.013846 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:19.337119 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:19.337078 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:19.535789 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:19.535760 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:20.216068 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:20.216043 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:28:20.217652 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:20.217628 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:28:22.704337 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:22.704280 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:22.785332 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:22.785299 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:22.847475 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:22.847434 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:22.947395 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:22.947362 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:23.267978 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:23.267944 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:23.369905 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:23.369860 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:27.237305 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:27.237264 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:27.564565 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:27.564533 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:27.686820 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:27.686782 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:28:27.883514 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:27.883419 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:28.481037 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:28.480998 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:30.738688 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:30.738651 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:30.838058 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:30.838016 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:30.935790 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:30.935744 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:31.039719 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:31.039637 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:31.517979 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:31.517942 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:31.618730 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:31.618689 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:35.265196 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:35.265162 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:36.392199 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:36.392152 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:28:38.495472 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:38.495436 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:40.384716 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.384680 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd"] Jul 05 12:28:40.385291 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.385267 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" containerName="main" Jul 05 12:28:40.385421 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.385305 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" containerName="main" Jul 05 12:28:40.385421 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.385357 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35350a7-092e-4be6-8264-c73556c4466a" containerName="main" Jul 05 12:28:40.385421 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.385367 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35350a7-092e-4be6-8264-c73556c4466a" containerName="main" Jul 05 12:28:40.385596 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.385489 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4e9c75b-1d2a-4230-8613-6b436dc8aa0a" containerName="main" Jul 05 12:28:40.385596 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.385504 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f35350a7-092e-4be6-8264-c73556c4466a" containerName="main" Jul 05 12:28:40.397052 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.397019 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.399141 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.399116 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd"] Jul 05 12:28:40.400136 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.400114 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6f19c8\"/\"openshift-service-ca.crt\"" Jul 05 12:28:40.400244 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.400114 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6f19c8\"/\"default-dockercfg-nsvt8\"" Jul 05 12:28:40.400445 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.400421 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6f19c8\"/\"kube-root-ca.crt\"" Jul 05 12:28:40.401475 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.401454 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6f19c8\"/\"auth-test-model-6f19c8-kserve-self-signed-certs\"" Jul 05 12:28:40.419759 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.419729 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:40.573802 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.573766 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfc017a8-c023-404e-b963-52ac6fe64db5-tls-certs\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.573802 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.573808 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-tmp-dir\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.574068 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.573922 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-home\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.574068 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.573990 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhbwc\" (UniqueName: \"kubernetes.io/projected/dfc017a8-c023-404e-b963-52ac6fe64db5-kube-api-access-mhbwc\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.574068 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.574047 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-model-cache\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.574068 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.574066 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-dshm\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.675123 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.675030 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-model-cache\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.675123 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.675066 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-dshm\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.675123 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.675088 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfc017a8-c023-404e-b963-52ac6fe64db5-tls-certs\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.675123 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.675109 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-tmp-dir\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.675461 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.675132 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-home\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.675461 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.675150 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhbwc\" (UniqueName: \"kubernetes.io/projected/dfc017a8-c023-404e-b963-52ac6fe64db5-kube-api-access-mhbwc\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.675561 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.675486 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-model-cache\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.675561 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.675547 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-tmp-dir\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.675630 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.675585 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-home\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.677545 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.677515 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-dshm\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.677737 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.677720 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfc017a8-c023-404e-b963-52ac6fe64db5-tls-certs\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.683321 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.683296 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhbwc\" (UniqueName: \"kubernetes.io/projected/dfc017a8-c023-404e-b963-52ac6fe64db5-kube-api-access-mhbwc\") pod \"auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.708059 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.708019 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:40.849722 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:40.849699 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd"] Jul 05 12:28:41.383855 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:41.383812 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" event={"ID":"dfc017a8-c023-404e-b963-52ac6fe64db5","Type":"ContainerStarted","Data":"1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f"} Jul 05 12:28:41.383855 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:41.383863 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" event={"ID":"dfc017a8-c023-404e-b963-52ac6fe64db5","Type":"ContainerStarted","Data":"eb6389002c6eb8128d94190f65027cdccbd7c981d357342f1d095b3d1dde58e1"} Jul 05 12:28:41.402213 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:41.402161 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" podStartSLOduration=1.40214704 podStartE2EDuration="1.40214704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:28:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:28:41.400807932 +0000 UTC m=+2121.791509695" watchObservedRunningTime="2026-07-05 12:28:41.40214704 +0000 UTC m=+2121.792848801" Jul 05 12:28:41.645383 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:41.645294 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:41.757221 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:41.757177 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:43.956806 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:43.956774 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:44.058533 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:44.058498 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:45.691823 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:45.691782 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:28:50.708382 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:50.708341 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:50.708382 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:50.708404 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:50.721247 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:50.721207 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:51.444832 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:51.444788 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:28:52.207506 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:52.207466 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:28:52.308236 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:28:52.308202 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:29:14.794264 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.794186 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f"] Jul 05 12:29:14.797821 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.797796 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.800537 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.800515 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"openshift-service-ca.crt\"" Jul 05 12:29:14.800642 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.800515 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"kube-root-ca.crt\"" Jul 05 12:29:14.801682 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.801668 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"default-dockercfg-msbcg\"" Jul 05 12:29:14.801770 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.801704 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"auth-test-model-85c29d-kserve-self-signed-certs\"" Jul 05 12:29:14.808547 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.808525 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f"] Jul 05 12:29:14.823222 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.823188 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:29:14.870021 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.869987 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzhqw\" (UniqueName: \"kubernetes.io/projected/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-kube-api-access-lzhqw\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.870210 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.870032 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tls-certs\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.870210 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.870051 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-home\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.870210 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.870136 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tmp-dir\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.870210 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.870176 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-dshm\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.870385 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.870237 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-model-cache\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.971193 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.971157 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-model-cache\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.971398 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.971251 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzhqw\" (UniqueName: \"kubernetes.io/projected/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-kube-api-access-lzhqw\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.971398 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.971280 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tls-certs\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.971398 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.971303 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-home\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.971398 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.971323 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tmp-dir\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.971398 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.971343 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-dshm\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.971673 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.971649 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-model-cache\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.971724 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.971685 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tmp-dir\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.971789 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.971728 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-home\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.973621 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.973601 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-dshm\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.973887 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.973856 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tls-certs\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:14.979064 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:14.979024 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzhqw\" (UniqueName: \"kubernetes.io/projected/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-kube-api-access-lzhqw\") pod \"auth-test-model-85c29d-kserve-7d4548c67-54g6f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:15.108459 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:15.108423 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:15.245767 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:15.245741 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f"] Jul 05 12:29:15.248353 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:29:15.248323 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb1117b5_d2cc_4f26_8e26_03cd24af8a3f.slice/crio-2d37fc563e10a428288faacd32053d9e606b25b7c1cddcbebb11e967e6e399bf WatchSource:0}: Error finding container 2d37fc563e10a428288faacd32053d9e606b25b7c1cddcbebb11e967e6e399bf: Status 404 returned error can't find the container with id 2d37fc563e10a428288faacd32053d9e606b25b7c1cddcbebb11e967e6e399bf Jul 05 12:29:15.568206 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:15.568168 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" event={"ID":"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f","Type":"ContainerStarted","Data":"08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf"} Jul 05 12:29:15.568206 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:15.568214 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" event={"ID":"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f","Type":"ContainerStarted","Data":"2d37fc563e10a428288faacd32053d9e606b25b7c1cddcbebb11e967e6e399bf"} Jul 05 12:29:15.583767 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:15.583714 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" podStartSLOduration=1.5836938329999999 podStartE2EDuration="1.583693833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:29:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:29:15.583449877 +0000 UTC m=+2155.974151641" watchObservedRunningTime="2026-07-05 12:29:15.583693833 +0000 UTC m=+2155.974395596" Jul 05 12:29:17.991099 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:17.991051 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:29:25.109562 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:25.109527 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:25.110015 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:25.109581 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:25.121971 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:25.121951 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:25.630541 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:25.630513 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:29:31.993818 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:31.993779 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:29:49.590889 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:29:49.590833 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:30:00.131294 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.131260 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29720910-czldf"] Jul 05 12:30:00.134523 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.134502 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt"] Jul 05 12:30:00.134624 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.134604 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720910-czldf" Jul 05 12:30:00.137265 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.137247 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-4cvrp\"" Jul 05 12:30:00.137721 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.137699 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" Jul 05 12:30:00.144149 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.144123 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720910-czldf"] Jul 05 12:30:00.159369 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.159333 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt"] Jul 05 12:30:00.281738 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.281695 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5bbm\" (UniqueName: \"kubernetes.io/projected/5c04386a-071a-4be3-b8a2-e2464ff95a72-kube-api-access-k5bbm\") pod \"maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt\" (UID: \"5c04386a-071a-4be3-b8a2-e2464ff95a72\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" Jul 05 12:30:00.281955 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.281763 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g76np\" (UniqueName: \"kubernetes.io/projected/a0b48d6b-57e8-4411-8d77-6a50cdec700d-kube-api-access-g76np\") pod \"maas-api-key-cleanup-29720910-czldf\" (UID: \"a0b48d6b-57e8-4411-8d77-6a50cdec700d\") " pod="opendatahub/maas-api-key-cleanup-29720910-czldf" Jul 05 12:30:00.382558 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.382474 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bbm\" (UniqueName: \"kubernetes.io/projected/5c04386a-071a-4be3-b8a2-e2464ff95a72-kube-api-access-k5bbm\") pod \"maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt\" (UID: \"5c04386a-071a-4be3-b8a2-e2464ff95a72\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" Jul 05 12:30:00.382558 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.382539 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g76np\" (UniqueName: \"kubernetes.io/projected/a0b48d6b-57e8-4411-8d77-6a50cdec700d-kube-api-access-g76np\") pod \"maas-api-key-cleanup-29720910-czldf\" (UID: \"a0b48d6b-57e8-4411-8d77-6a50cdec700d\") " pod="opendatahub/maas-api-key-cleanup-29720910-czldf" Jul 05 12:30:00.390736 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.390700 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g76np\" (UniqueName: \"kubernetes.io/projected/a0b48d6b-57e8-4411-8d77-6a50cdec700d-kube-api-access-g76np\") pod \"maas-api-key-cleanup-29720910-czldf\" (UID: \"a0b48d6b-57e8-4411-8d77-6a50cdec700d\") " pod="opendatahub/maas-api-key-cleanup-29720910-czldf" Jul 05 12:30:00.390892 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.390704 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5bbm\" (UniqueName: \"kubernetes.io/projected/5c04386a-071a-4be3-b8a2-e2464ff95a72-kube-api-access-k5bbm\") pod \"maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt\" (UID: \"5c04386a-071a-4be3-b8a2-e2464ff95a72\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" Jul 05 12:30:00.447285 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.447246 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720910-czldf" Jul 05 12:30:00.456409 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.456382 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" Jul 05 12:30:00.598402 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.598372 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720910-czldf"] Jul 05 12:30:00.599936 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:30:00.599903 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b48d6b_57e8_4411_8d77_6a50cdec700d.slice/crio-bffe4ad88515d5e0a2ffdb28bce1c599fd14e42cfd4435755ad1de8352888cc0 WatchSource:0}: Error finding container bffe4ad88515d5e0a2ffdb28bce1c599fd14e42cfd4435755ad1de8352888cc0: Status 404 returned error can't find the container with id bffe4ad88515d5e0a2ffdb28bce1c599fd14e42cfd4435755ad1de8352888cc0 Jul 05 12:30:00.602081 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.602061 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 12:30:00.624172 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.624149 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt"] Jul 05 12:30:00.626346 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:30:00.626319 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c04386a_071a_4be3_b8a2_e2464ff95a72.slice/crio-c97d365109e42ce4fbded8a84450593ecd5d03124559a53d0948aab2e0344981 WatchSource:0}: Error finding container c97d365109e42ce4fbded8a84450593ecd5d03124559a53d0948aab2e0344981: Status 404 returned error can't find the container with id c97d365109e42ce4fbded8a84450593ecd5d03124559a53d0948aab2e0344981 Jul 05 12:30:00.799748 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.799649 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" event={"ID":"5c04386a-071a-4be3-b8a2-e2464ff95a72","Type":"ContainerStarted","Data":"c97d365109e42ce4fbded8a84450593ecd5d03124559a53d0948aab2e0344981"} Jul 05 12:30:00.800895 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:00.800850 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720910-czldf" event={"ID":"a0b48d6b-57e8-4411-8d77-6a50cdec700d","Type":"ContainerStarted","Data":"bffe4ad88515d5e0a2ffdb28bce1c599fd14e42cfd4435755ad1de8352888cc0"} Jul 05 12:30:03.822742 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:03.822703 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" event={"ID":"5c04386a-071a-4be3-b8a2-e2464ff95a72","Type":"ContainerStarted","Data":"811d47f9fc72e3824b1b282dd1cf91ad02c726a9afb46f41375e44d815d2b3f6"} Jul 05 12:30:03.824282 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:03.824257 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720910-czldf" event={"ID":"a0b48d6b-57e8-4411-8d77-6a50cdec700d","Type":"ContainerStarted","Data":"0be193ae6363e89df475900ec22404ee2d05be70297583819d7e88f21e97f3d9"} Jul 05 12:30:03.838820 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:03.838757 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" podStartSLOduration=1.334459283 podStartE2EDuration="3.838738184s" totalImagesPullingTime="2.504278901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:30:03.837169808 +0000 UTC m=+2204.227871570" watchObservedRunningTime="2026-07-05 12:30:03.838738184 +0000 UTC m=+2204.229439949" Jul 05 12:30:03.851660 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:03.851609 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29720910-czldf" podStartSLOduration=1.467405393 podStartE2EDuration="3.8515936s" totalImagesPullingTime="2.384188207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:30:03.850412044 +0000 UTC m=+2204.241113808" watchObservedRunningTime="2026-07-05 12:30:03.8515936 +0000 UTC m=+2204.242295361" Jul 05 12:30:04.830534 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:04.830498 2575 generic.go:356] "Generic (PLEG): container finished" podID="5c04386a-071a-4be3-b8a2-e2464ff95a72" containerID="811d47f9fc72e3824b1b282dd1cf91ad02c726a9afb46f41375e44d815d2b3f6" exitCode=0 Jul 05 12:30:04.831005 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:04.830590 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" event={"ID":"5c04386a-071a-4be3-b8a2-e2464ff95a72","Type":"ContainerDied","Data":"811d47f9fc72e3824b1b282dd1cf91ad02c726a9afb46f41375e44d815d2b3f6"} Jul 05 12:30:04.831939 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:04.831917 2575 generic.go:356] "Generic (PLEG): container finished" podID="a0b48d6b-57e8-4411-8d77-6a50cdec700d" containerID="0be193ae6363e89df475900ec22404ee2d05be70297583819d7e88f21e97f3d9" exitCode=0 Jul 05 12:30:04.832006 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:04.831987 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720910-czldf" event={"ID":"a0b48d6b-57e8-4411-8d77-6a50cdec700d","Type":"ContainerDied","Data":"0be193ae6363e89df475900ec22404ee2d05be70297583819d7e88f21e97f3d9"} Jul 05 12:30:05.699662 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:05.699627 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:30:05.994116 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:05.994085 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" Jul 05 12:30:06.028934 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.028905 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720910-czldf" Jul 05 12:30:06.135959 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.135934 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5bbm\" (UniqueName: \"kubernetes.io/projected/5c04386a-071a-4be3-b8a2-e2464ff95a72-kube-api-access-k5bbm\") pod \"5c04386a-071a-4be3-b8a2-e2464ff95a72\" (UID: \"5c04386a-071a-4be3-b8a2-e2464ff95a72\") " Jul 05 12:30:06.136131 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.135979 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g76np\" (UniqueName: \"kubernetes.io/projected/a0b48d6b-57e8-4411-8d77-6a50cdec700d-kube-api-access-g76np\") pod \"a0b48d6b-57e8-4411-8d77-6a50cdec700d\" (UID: \"a0b48d6b-57e8-4411-8d77-6a50cdec700d\") " Jul 05 12:30:06.138226 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.138192 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c04386a-071a-4be3-b8a2-e2464ff95a72-kube-api-access-k5bbm" (OuterVolumeSpecName: "kube-api-access-k5bbm") pod "5c04386a-071a-4be3-b8a2-e2464ff95a72" (UID: "5c04386a-071a-4be3-b8a2-e2464ff95a72"). InnerVolumeSpecName "kube-api-access-k5bbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:30:06.138324 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.138218 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0b48d6b-57e8-4411-8d77-6a50cdec700d-kube-api-access-g76np" (OuterVolumeSpecName: "kube-api-access-g76np") pod "a0b48d6b-57e8-4411-8d77-6a50cdec700d" (UID: "a0b48d6b-57e8-4411-8d77-6a50cdec700d"). InnerVolumeSpecName "kube-api-access-g76np". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:30:06.237633 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.237545 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5bbm\" (UniqueName: \"kubernetes.io/projected/5c04386a-071a-4be3-b8a2-e2464ff95a72-kube-api-access-k5bbm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:30:06.237633 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.237573 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g76np\" (UniqueName: \"kubernetes.io/projected/a0b48d6b-57e8-4411-8d77-6a50cdec700d-kube-api-access-g76np\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:30:06.844803 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.844769 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720910-czldf" Jul 05 12:30:06.844997 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.844803 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720910-czldf" event={"ID":"a0b48d6b-57e8-4411-8d77-6a50cdec700d","Type":"ContainerDied","Data":"bffe4ad88515d5e0a2ffdb28bce1c599fd14e42cfd4435755ad1de8352888cc0"} Jul 05 12:30:06.844997 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.844914 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bffe4ad88515d5e0a2ffdb28bce1c599fd14e42cfd4435755ad1de8352888cc0" Jul 05 12:30:06.846305 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.846280 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" event={"ID":"5c04386a-071a-4be3-b8a2-e2464ff95a72","Type":"ContainerDied","Data":"c97d365109e42ce4fbded8a84450593ecd5d03124559a53d0948aab2e0344981"} Jul 05 12:30:06.846428 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.846319 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c97d365109e42ce4fbded8a84450593ecd5d03124559a53d0948aab2e0344981" Jul 05 12:30:06.846428 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:06.846329 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt" Jul 05 12:30:18.794164 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:18.794125 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:30:34.695248 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:34.695207 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:30:49.993525 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:49.993441 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:30:59.692604 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:59.692558 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd"] Jul 05 12:30:59.737864 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:30:59.737836 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:31:00.574253 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:00.574207 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f"] Jul 05 12:31:00.828589 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:00.828501 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:31:04.599148 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:04.599111 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:31:10.794698 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.794664 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc"] Jul 05 12:31:10.795117 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.795084 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0b48d6b-57e8-4411-8d77-6a50cdec700d" containerName="cleanup" Jul 05 12:31:10.795117 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.795100 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0b48d6b-57e8-4411-8d77-6a50cdec700d" containerName="cleanup" Jul 05 12:31:10.795117 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.795119 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c04386a-071a-4be3-b8a2-e2464ff95a72" containerName="cleanup" Jul 05 12:31:10.795223 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.795125 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c04386a-071a-4be3-b8a2-e2464ff95a72" containerName="cleanup" Jul 05 12:31:10.795223 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.795189 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c04386a-071a-4be3-b8a2-e2464ff95a72" containerName="cleanup" Jul 05 12:31:10.795223 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.795201 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0b48d6b-57e8-4411-8d77-6a50cdec700d" containerName="cleanup" Jul 05 12:31:10.798442 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.798424 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.800927 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.800902 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6f19c8\"/\"sub-test-model-6f19c8-kserve-self-signed-certs\"" Jul 05 12:31:10.805544 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.805521 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc"] Jul 05 12:31:10.813930 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.813903 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-model-cache\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.814044 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.813949 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-tmp-dir\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.814044 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.813971 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2hlm\" (UniqueName: \"kubernetes.io/projected/59eb9e58-ccec-4686-9e32-8d51b855c557-kube-api-access-x2hlm\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.814129 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.814101 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59eb9e58-ccec-4686-9e32-8d51b855c557-tls-certs\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.814167 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.814149 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-dshm\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.814202 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.814191 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-home\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.823552 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.823528 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:31:10.914970 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.914939 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-dshm\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.915177 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.914977 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-home\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.915177 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.915043 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-model-cache\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.915177 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.915074 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-tmp-dir\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.915177 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.915095 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2hlm\" (UniqueName: \"kubernetes.io/projected/59eb9e58-ccec-4686-9e32-8d51b855c557-kube-api-access-x2hlm\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.915177 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.915144 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59eb9e58-ccec-4686-9e32-8d51b855c557-tls-certs\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.915518 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.915489 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-model-cache\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.915653 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.915525 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-home\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.915653 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.915590 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-tmp-dir\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.917255 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.917235 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-dshm\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.917572 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.917552 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59eb9e58-ccec-4686-9e32-8d51b855c557-tls-certs\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:10.923716 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:10.923693 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2hlm\" (UniqueName: \"kubernetes.io/projected/59eb9e58-ccec-4686-9e32-8d51b855c557-kube-api-access-x2hlm\") pod \"sub-test-model-6f19c8-kserve-fd5d576f-znwjc\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:11.108745 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:11.108713 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:11.248737 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:11.248706 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc"] Jul 05 12:31:11.251404 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:31:11.251370 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59eb9e58_ccec_4686_9e32_8d51b855c557.slice/crio-3f2929d78f1112e1f637e855f370b2b88145b1c1626aa5627993f64e4fcb5380 WatchSource:0}: Error finding container 3f2929d78f1112e1f637e855f370b2b88145b1c1626aa5627993f64e4fcb5380: Status 404 returned error can't find the container with id 3f2929d78f1112e1f637e855f370b2b88145b1c1626aa5627993f64e4fcb5380 Jul 05 12:31:12.186900 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:12.186841 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" event={"ID":"59eb9e58-ccec-4686-9e32-8d51b855c557","Type":"ContainerStarted","Data":"116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0"} Jul 05 12:31:12.186900 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:12.186906 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" event={"ID":"59eb9e58-ccec-4686-9e32-8d51b855c557","Type":"ContainerStarted","Data":"3f2929d78f1112e1f637e855f370b2b88145b1c1626aa5627993f64e4fcb5380"} Jul 05 12:31:12.202572 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:12.202520 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" podStartSLOduration=2.202505229 podStartE2EDuration="2.202505229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:31:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:31:12.201736318 +0000 UTC m=+2272.592438080" watchObservedRunningTime="2026-07-05 12:31:12.202505229 +0000 UTC m=+2272.593206991" Jul 05 12:31:14.713284 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:14.713246 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" podUID="dfc017a8-c023-404e-b963-52ac6fe64db5" containerName="main" containerID="cri-o://1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f" gracePeriod=45 Jul 05 12:31:14.969298 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:14.969223 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:31:15.047941 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.047906 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhbwc\" (UniqueName: \"kubernetes.io/projected/dfc017a8-c023-404e-b963-52ac6fe64db5-kube-api-access-mhbwc\") pod \"dfc017a8-c023-404e-b963-52ac6fe64db5\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " Jul 05 12:31:15.048141 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.047960 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-home\") pod \"dfc017a8-c023-404e-b963-52ac6fe64db5\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " Jul 05 12:31:15.048141 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.048012 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-model-cache\") pod \"dfc017a8-c023-404e-b963-52ac6fe64db5\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " Jul 05 12:31:15.048141 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.048063 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-dshm\") pod \"dfc017a8-c023-404e-b963-52ac6fe64db5\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " Jul 05 12:31:15.048141 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.048127 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-tmp-dir\") pod \"dfc017a8-c023-404e-b963-52ac6fe64db5\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " Jul 05 12:31:15.048397 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.048166 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfc017a8-c023-404e-b963-52ac6fe64db5-tls-certs\") pod \"dfc017a8-c023-404e-b963-52ac6fe64db5\" (UID: \"dfc017a8-c023-404e-b963-52ac6fe64db5\") " Jul 05 12:31:15.048397 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.048293 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-model-cache" (OuterVolumeSpecName: "model-cache") pod "dfc017a8-c023-404e-b963-52ac6fe64db5" (UID: "dfc017a8-c023-404e-b963-52ac6fe64db5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:31:15.048397 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.048258 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-home" (OuterVolumeSpecName: "home") pod "dfc017a8-c023-404e-b963-52ac6fe64db5" (UID: "dfc017a8-c023-404e-b963-52ac6fe64db5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:31:15.048551 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.048456 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dfc017a8-c023-404e-b963-52ac6fe64db5" (UID: "dfc017a8-c023-404e-b963-52ac6fe64db5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:31:15.048551 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.048479 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-home\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:15.048551 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.048497 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-model-cache\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:15.050340 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.050316 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-dshm" (OuterVolumeSpecName: "dshm") pod "dfc017a8-c023-404e-b963-52ac6fe64db5" (UID: "dfc017a8-c023-404e-b963-52ac6fe64db5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:31:15.050340 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.050332 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfc017a8-c023-404e-b963-52ac6fe64db5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dfc017a8-c023-404e-b963-52ac6fe64db5" (UID: "dfc017a8-c023-404e-b963-52ac6fe64db5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:31:15.050487 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.050444 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfc017a8-c023-404e-b963-52ac6fe64db5-kube-api-access-mhbwc" (OuterVolumeSpecName: "kube-api-access-mhbwc") pod "dfc017a8-c023-404e-b963-52ac6fe64db5" (UID: "dfc017a8-c023-404e-b963-52ac6fe64db5"). InnerVolumeSpecName "kube-api-access-mhbwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:31:15.149947 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.149916 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfc017a8-c023-404e-b963-52ac6fe64db5-tls-certs\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:15.149947 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.149944 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhbwc\" (UniqueName: \"kubernetes.io/projected/dfc017a8-c023-404e-b963-52ac6fe64db5-kube-api-access-mhbwc\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:15.149947 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.149954 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-dshm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:15.150182 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.149964 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfc017a8-c023-404e-b963-52ac6fe64db5-tmp-dir\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:15.202717 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.202687 2575 generic.go:356] "Generic (PLEG): container finished" podID="dfc017a8-c023-404e-b963-52ac6fe64db5" containerID="1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f" exitCode=0 Jul 05 12:31:15.202916 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.202728 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" Jul 05 12:31:15.202916 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.202779 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" event={"ID":"dfc017a8-c023-404e-b963-52ac6fe64db5","Type":"ContainerDied","Data":"1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f"} Jul 05 12:31:15.202916 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.202833 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd" event={"ID":"dfc017a8-c023-404e-b963-52ac6fe64db5","Type":"ContainerDied","Data":"eb6389002c6eb8128d94190f65027cdccbd7c981d357342f1d095b3d1dde58e1"} Jul 05 12:31:15.202916 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.202856 2575 scope.go:117] "RemoveContainer" containerID="1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f" Jul 05 12:31:15.212441 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.212420 2575 scope.go:117] "RemoveContainer" containerID="1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f" Jul 05 12:31:15.212699 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:31:15.212680 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f\": container with ID starting with 1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f not found: ID does not exist" containerID="1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f" Jul 05 12:31:15.212766 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.212708 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f"} err="failed to get container status \"1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f\": rpc error: code = NotFound desc = could not find container \"1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f\": container with ID starting with 1cb10865dcdbd77bee64f5d02be96d0c4903c4ce3e509f96e5391f798600763f not found: ID does not exist" Jul 05 12:31:15.224514 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.224460 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd"] Jul 05 12:31:15.230082 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.230057 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/auth-test-model-6f19c8-kserve-5d5bc996c5-h5fmd"] Jul 05 12:31:15.594984 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.594935 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" podUID="cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" containerName="main" containerID="cri-o://08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf" gracePeriod=45 Jul 05 12:31:15.620489 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.620451 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" podUID="cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": dial tcp 10.132.0.46:8000: connect: connection refused" Jul 05 12:31:15.853002 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.852937 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:31:15.957935 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.957901 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-model-cache\") pod \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " Jul 05 12:31:15.958125 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.957960 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tmp-dir\") pod \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " Jul 05 12:31:15.958125 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.957998 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzhqw\" (UniqueName: \"kubernetes.io/projected/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-kube-api-access-lzhqw\") pod \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " Jul 05 12:31:15.958125 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.958041 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-home\") pod \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " Jul 05 12:31:15.958125 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.958102 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tls-certs\") pod \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " Jul 05 12:31:15.958353 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.958143 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-dshm\") pod \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\" (UID: \"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f\") " Jul 05 12:31:15.958353 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.958213 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-model-cache" (OuterVolumeSpecName: "model-cache") pod "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" (UID: "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:31:15.958353 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.958271 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" (UID: "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:31:15.958483 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.958352 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-home" (OuterVolumeSpecName: "home") pod "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" (UID: "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:31:15.958483 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.958409 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-model-cache\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:15.958483 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.958428 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tmp-dir\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:15.960282 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.960259 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-dshm" (OuterVolumeSpecName: "dshm") pod "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" (UID: "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:31:15.960367 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.960300 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-kube-api-access-lzhqw" (OuterVolumeSpecName: "kube-api-access-lzhqw") pod "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" (UID: "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f"). InnerVolumeSpecName "kube-api-access-lzhqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:31:15.960367 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:15.960358 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" (UID: "cb1117b5-d2cc-4f26-8e26-03cd24af8a3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:31:16.059069 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.059031 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-tls-certs\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:16.059069 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.059063 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-dshm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:16.059069 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.059073 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzhqw\" (UniqueName: \"kubernetes.io/projected/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-kube-api-access-lzhqw\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:16.059069 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.059081 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f-home\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:31:16.130393 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.130307 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfc017a8-c023-404e-b963-52ac6fe64db5" path="/var/lib/kubelet/pods/dfc017a8-c023-404e-b963-52ac6fe64db5/volumes" Jul 05 12:31:16.209217 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.209181 2575 generic.go:356] "Generic (PLEG): container finished" podID="cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" containerID="08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf" exitCode=0 Jul 05 12:31:16.209408 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.209225 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" Jul 05 12:31:16.209408 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.209263 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" event={"ID":"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f","Type":"ContainerDied","Data":"08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf"} Jul 05 12:31:16.209408 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.209306 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f" event={"ID":"cb1117b5-d2cc-4f26-8e26-03cd24af8a3f","Type":"ContainerDied","Data":"2d37fc563e10a428288faacd32053d9e606b25b7c1cddcbebb11e967e6e399bf"} Jul 05 12:31:16.209408 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.209327 2575 scope.go:117] "RemoveContainer" containerID="08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf" Jul 05 12:31:16.217820 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.217800 2575 scope.go:117] "RemoveContainer" containerID="08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf" Jul 05 12:31:16.218088 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:31:16.218067 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf\": container with ID starting with 08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf not found: ID does not exist" containerID="08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf" Jul 05 12:31:16.218160 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.218102 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf"} err="failed to get container status \"08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf\": rpc error: code = NotFound desc = could not find container \"08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf\": container with ID starting with 08c9b15b41de4bd3fe68d2932c34266de255fc3d6d560d722f299d4858902dbf not found: ID does not exist" Jul 05 12:31:16.226326 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.226298 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f"] Jul 05 12:31:16.228006 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:16.227988 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/auth-test-model-85c29d-kserve-7d4548c67-54g6f"] Jul 05 12:31:18.129502 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:18.129470 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" path="/var/lib/kubelet/pods/cb1117b5-d2cc-4f26-8e26-03cd24af8a3f/volumes" Jul 05 12:31:21.109061 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:21.109021 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:21.109565 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:21.109075 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:21.122011 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:21.121974 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:21.246654 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:21.246626 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:31:22.028854 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:22.028817 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:31:22.083922 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:22.083862 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:31:23.108953 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:23.108916 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:31:36.301143 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.301105 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng"] Jul 05 12:31:36.301781 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.301760 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfc017a8-c023-404e-b963-52ac6fe64db5" containerName="main" Jul 05 12:31:36.301843 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.301785 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfc017a8-c023-404e-b963-52ac6fe64db5" containerName="main" Jul 05 12:31:36.301843 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.301811 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" containerName="main" Jul 05 12:31:36.301843 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.301820 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" containerName="main" Jul 05 12:31:36.301968 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.301904 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfc017a8-c023-404e-b963-52ac6fe64db5" containerName="main" Jul 05 12:31:36.301968 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.301918 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb1117b5-d2cc-4f26-8e26-03cd24af8a3f" containerName="main" Jul 05 12:31:36.304900 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.304856 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.308893 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.308856 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"openshift-service-ca.crt\"" Jul 05 12:31:36.309024 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.308903 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"default-dockercfg-msbcg\"" Jul 05 12:31:36.309024 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.308910 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"sub-test-model-85c29d-kserve-self-signed-certs\"" Jul 05 12:31:36.309024 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.308910 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"kube-root-ca.crt\"" Jul 05 12:31:36.314809 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.314774 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng"] Jul 05 12:31:36.329805 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.329778 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:31:36.437349 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.437305 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tls-certs\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.437542 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.437376 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-model-cache\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.437542 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.437451 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r2cb\" (UniqueName: \"kubernetes.io/projected/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-kube-api-access-8r2cb\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.437658 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.437539 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-home\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.437658 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.437576 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-dshm\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.437658 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.437607 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tmp-dir\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.538393 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.538359 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tls-certs\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.538585 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.538402 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-model-cache\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.538585 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.538428 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r2cb\" (UniqueName: \"kubernetes.io/projected/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-kube-api-access-8r2cb\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.538585 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.538475 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-home\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.538585 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.538510 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-dshm\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.538585 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.538540 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tmp-dir\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.538931 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.538857 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-model-cache\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.538931 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.538890 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-home\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.539216 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.539197 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tmp-dir\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.541185 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.541156 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-dshm\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.541295 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.541158 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tls-certs\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.546165 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.546142 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r2cb\" (UniqueName: \"kubernetes.io/projected/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-kube-api-access-8r2cb\") pod \"sub-test-model-85c29d-kserve-5c57bb7769-56bng\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.615832 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.615792 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:36.757001 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:36.756973 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng"] Jul 05 12:31:36.758561 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:31:36.758536 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c14fcdc_14c4_43b3_8d4f_2b77e37aab9b.slice/crio-76a79481eace880b7630e7acb9d84ad3949f284aed597b9db7ca476574388e59 WatchSource:0}: Error finding container 76a79481eace880b7630e7acb9d84ad3949f284aed597b9db7ca476574388e59: Status 404 returned error can't find the container with id 76a79481eace880b7630e7acb9d84ad3949f284aed597b9db7ca476574388e59 Jul 05 12:31:37.314761 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:37.314721 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" event={"ID":"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b","Type":"ContainerStarted","Data":"b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef"} Jul 05 12:31:37.314761 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:37.314770 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" event={"ID":"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b","Type":"ContainerStarted","Data":"76a79481eace880b7630e7acb9d84ad3949f284aed597b9db7ca476574388e59"} Jul 05 12:31:37.332230 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:37.332181 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" podStartSLOduration=1.3321665679999999 podStartE2EDuration="1.332166568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:31:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:31:37.331412047 +0000 UTC m=+2297.722113813" watchObservedRunningTime="2026-07-05 12:31:37.332166568 +0000 UTC m=+2297.722868329" Jul 05 12:31:40.503740 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:40.503700 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:31:40.553452 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:40.553406 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:31:40.653753 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:40.653699 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:31:46.616128 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:46.616081 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:46.616577 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:46.616143 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:46.629113 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:46.629082 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:31:47.374804 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:31:47.374773 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:32:08.187304 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:08.187264 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:32:24.488712 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:24.488633 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:32:28.995356 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:28.995321 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc"] Jul 05 12:32:29.019518 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:29.019484 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:32:29.971365 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:29.971324 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng"] Jul 05 12:32:29.993292 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:29.993257 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:32:38.193560 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.193526 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p"] Jul 05 12:32:38.197002 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.196984 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.199927 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.199903 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6f19c8\"/\"rate-test-model-6f19c8-kserve-self-signed-certs\"" Jul 05 12:32:38.205045 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.205017 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p"] Jul 05 12:32:38.217751 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.217717 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:32:38.298074 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.298039 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3158c734-327e-454d-bb5a-1c13f3d1d445-tls-certs\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.298242 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.298201 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-tmp-dir\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.298441 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.298242 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk7nw\" (UniqueName: \"kubernetes.io/projected/3158c734-327e-454d-bb5a-1c13f3d1d445-kube-api-access-gk7nw\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.298441 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.298281 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-dshm\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.298441 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.298311 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-model-cache\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.298441 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.298345 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-home\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.398850 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.398805 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-tmp-dir\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.398850 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.398851 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk7nw\" (UniqueName: \"kubernetes.io/projected/3158c734-327e-454d-bb5a-1c13f3d1d445-kube-api-access-gk7nw\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.399113 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.398893 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-dshm\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.399113 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.398912 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-model-cache\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.399113 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.398944 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-home\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.399113 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.398971 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3158c734-327e-454d-bb5a-1c13f3d1d445-tls-certs\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.399324 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.399302 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-tmp-dir\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.399398 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.399363 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-model-cache\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.399510 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.399399 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-home\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.401331 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.401303 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-dshm\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.401442 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.401416 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3158c734-327e-454d-bb5a-1c13f3d1d445-tls-certs\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.406685 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.406661 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk7nw\" (UniqueName: \"kubernetes.io/projected/3158c734-327e-454d-bb5a-1c13f3d1d445-kube-api-access-gk7nw\") pod \"rate-test-model-6f19c8-kserve-58d5d777d7-h878p\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.506744 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.506643 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:38.643352 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:38.643317 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p"] Jul 05 12:32:38.645330 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:32:38.645299 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3158c734_327e_454d_bb5a_1c13f3d1d445.slice/crio-62ef5f8f8a040453f7fc36752f3da3cd56eb624befb06f8325c654ae8720ceb0 WatchSource:0}: Error finding container 62ef5f8f8a040453f7fc36752f3da3cd56eb624befb06f8325c654ae8720ceb0: Status 404 returned error can't find the container with id 62ef5f8f8a040453f7fc36752f3da3cd56eb624befb06f8325c654ae8720ceb0 Jul 05 12:32:39.615508 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:39.615470 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" event={"ID":"3158c734-327e-454d-bb5a-1c13f3d1d445","Type":"ContainerStarted","Data":"51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da"} Jul 05 12:32:39.615993 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:39.615515 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" event={"ID":"3158c734-327e-454d-bb5a-1c13f3d1d445","Type":"ContainerStarted","Data":"62ef5f8f8a040453f7fc36752f3da3cd56eb624befb06f8325c654ae8720ceb0"} Jul 05 12:32:39.633764 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:39.633717 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" podStartSLOduration=1.633702901 podStartE2EDuration="1.633702901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:32:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:32:39.631443184 +0000 UTC m=+2360.022144947" watchObservedRunningTime="2026-07-05 12:32:39.633702901 +0000 UTC m=+2360.024404722" Jul 05 12:32:40.792330 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:40.792288 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:32:44.015631 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.015564 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" podUID="59eb9e58-ccec-4686-9e32-8d51b855c557" containerName="main" containerID="cri-o://116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0" gracePeriod=45 Jul 05 12:32:44.269805 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.269739 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:32:44.345040 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.344999 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-tmp-dir\") pod \"59eb9e58-ccec-4686-9e32-8d51b855c557\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " Jul 05 12:32:44.345040 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.345040 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-dshm\") pod \"59eb9e58-ccec-4686-9e32-8d51b855c557\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " Jul 05 12:32:44.345305 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.345071 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2hlm\" (UniqueName: \"kubernetes.io/projected/59eb9e58-ccec-4686-9e32-8d51b855c557-kube-api-access-x2hlm\") pod \"59eb9e58-ccec-4686-9e32-8d51b855c557\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " Jul 05 12:32:44.345305 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.345097 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-home\") pod \"59eb9e58-ccec-4686-9e32-8d51b855c557\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " Jul 05 12:32:44.345305 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.345149 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59eb9e58-ccec-4686-9e32-8d51b855c557-tls-certs\") pod \"59eb9e58-ccec-4686-9e32-8d51b855c557\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " Jul 05 12:32:44.345305 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.345189 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-model-cache\") pod \"59eb9e58-ccec-4686-9e32-8d51b855c557\" (UID: \"59eb9e58-ccec-4686-9e32-8d51b855c557\") " Jul 05 12:32:44.345514 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.345339 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59eb9e58-ccec-4686-9e32-8d51b855c557" (UID: "59eb9e58-ccec-4686-9e32-8d51b855c557"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:32:44.345514 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.345348 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-home" (OuterVolumeSpecName: "home") pod "59eb9e58-ccec-4686-9e32-8d51b855c557" (UID: "59eb9e58-ccec-4686-9e32-8d51b855c557"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:32:44.345514 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.345431 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-tmp-dir\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:44.345514 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.345450 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-home\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:44.345674 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.345560 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-model-cache" (OuterVolumeSpecName: "model-cache") pod "59eb9e58-ccec-4686-9e32-8d51b855c557" (UID: "59eb9e58-ccec-4686-9e32-8d51b855c557"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:32:44.347296 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.347270 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-dshm" (OuterVolumeSpecName: "dshm") pod "59eb9e58-ccec-4686-9e32-8d51b855c557" (UID: "59eb9e58-ccec-4686-9e32-8d51b855c557"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:32:44.347416 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.347381 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59eb9e58-ccec-4686-9e32-8d51b855c557-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59eb9e58-ccec-4686-9e32-8d51b855c557" (UID: "59eb9e58-ccec-4686-9e32-8d51b855c557"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:32:44.347494 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.347471 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59eb9e58-ccec-4686-9e32-8d51b855c557-kube-api-access-x2hlm" (OuterVolumeSpecName: "kube-api-access-x2hlm") pod "59eb9e58-ccec-4686-9e32-8d51b855c557" (UID: "59eb9e58-ccec-4686-9e32-8d51b855c557"). InnerVolumeSpecName "kube-api-access-x2hlm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:32:44.446220 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.446190 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59eb9e58-ccec-4686-9e32-8d51b855c557-tls-certs\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:44.446220 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.446219 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-model-cache\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:44.446220 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.446228 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59eb9e58-ccec-4686-9e32-8d51b855c557-dshm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:44.446463 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.446238 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2hlm\" (UniqueName: \"kubernetes.io/projected/59eb9e58-ccec-4686-9e32-8d51b855c557-kube-api-access-x2hlm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:44.643410 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.643373 2575 generic.go:356] "Generic (PLEG): container finished" podID="59eb9e58-ccec-4686-9e32-8d51b855c557" containerID="116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0" exitCode=0 Jul 05 12:32:44.643580 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.643428 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" Jul 05 12:32:44.643580 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.643464 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" event={"ID":"59eb9e58-ccec-4686-9e32-8d51b855c557","Type":"ContainerDied","Data":"116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0"} Jul 05 12:32:44.643580 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.643511 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc" event={"ID":"59eb9e58-ccec-4686-9e32-8d51b855c557","Type":"ContainerDied","Data":"3f2929d78f1112e1f637e855f370b2b88145b1c1626aa5627993f64e4fcb5380"} Jul 05 12:32:44.643580 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.643529 2575 scope.go:117] "RemoveContainer" containerID="116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0" Jul 05 12:32:44.652799 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.652784 2575 scope.go:117] "RemoveContainer" containerID="116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0" Jul 05 12:32:44.653098 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:32:44.653077 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0\": container with ID starting with 116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0 not found: ID does not exist" containerID="116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0" Jul 05 12:32:44.653159 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.653107 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0"} err="failed to get container status \"116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0\": rpc error: code = NotFound desc = could not find container \"116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0\": container with ID starting with 116a8e96ffc58a89922d095272e2b0a6561c7bc560f55fc9db3076768d25baf0 not found: ID does not exist" Jul 05 12:32:44.666631 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.666604 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc"] Jul 05 12:32:44.671247 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.671225 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/sub-test-model-6f19c8-kserve-fd5d576f-znwjc"] Jul 05 12:32:44.991862 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:44.991762 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" podUID="3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" containerName="main" containerID="cri-o://b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef" gracePeriod=45 Jul 05 12:32:45.260155 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.260126 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:32:45.352524 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.352480 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tmp-dir\") pod \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " Jul 05 12:32:45.352728 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.352543 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tls-certs\") pod \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " Jul 05 12:32:45.352728 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.352585 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-home\") pod \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " Jul 05 12:32:45.352728 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.352610 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-dshm\") pod \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " Jul 05 12:32:45.352728 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.352648 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r2cb\" (UniqueName: \"kubernetes.io/projected/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-kube-api-access-8r2cb\") pod \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " Jul 05 12:32:45.352728 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.352698 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-model-cache\") pod \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\" (UID: \"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b\") " Jul 05 12:32:45.352994 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.352743 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" (UID: "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:32:45.352994 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.352891 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-home" (OuterVolumeSpecName: "home") pod "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" (UID: "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:32:45.353111 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.353078 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-home\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:45.353111 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.353081 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-model-cache" (OuterVolumeSpecName: "model-cache") pod "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" (UID: "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:32:45.353111 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.353097 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tmp-dir\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:45.354932 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.354900 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-dshm" (OuterVolumeSpecName: "dshm") pod "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" (UID: "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:32:45.354932 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.354915 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-kube-api-access-8r2cb" (OuterVolumeSpecName: "kube-api-access-8r2cb") pod "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" (UID: "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b"). InnerVolumeSpecName "kube-api-access-8r2cb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:32:45.355098 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.354990 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" (UID: "3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:32:45.453592 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.453553 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-tls-certs\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:45.453592 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.453586 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-dshm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:45.453592 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.453595 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r2cb\" (UniqueName: \"kubernetes.io/projected/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-kube-api-access-8r2cb\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:45.453592 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.453604 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b-model-cache\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:32:45.656004 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.655969 2575 generic.go:356] "Generic (PLEG): container finished" podID="3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" containerID="b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef" exitCode=0 Jul 05 12:32:45.656207 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.656020 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" Jul 05 12:32:45.656207 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.656055 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" event={"ID":"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b","Type":"ContainerDied","Data":"b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef"} Jul 05 12:32:45.656207 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.656097 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng" event={"ID":"3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b","Type":"ContainerDied","Data":"76a79481eace880b7630e7acb9d84ad3949f284aed597b9db7ca476574388e59"} Jul 05 12:32:45.656207 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.656114 2575 scope.go:117] "RemoveContainer" containerID="b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef" Jul 05 12:32:45.665345 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.665328 2575 scope.go:117] "RemoveContainer" containerID="b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef" Jul 05 12:32:45.665623 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:32:45.665601 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef\": container with ID starting with b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef not found: ID does not exist" containerID="b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef" Jul 05 12:32:45.665697 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.665634 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef"} err="failed to get container status \"b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef\": rpc error: code = NotFound desc = could not find container \"b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef\": container with ID starting with b2b340e9fb93948934ac146ada6d06a089334b629f150de9c718c171344db6ef not found: ID does not exist" Jul 05 12:32:45.677325 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.677303 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng"] Jul 05 12:32:45.683353 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:45.683328 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/sub-test-model-85c29d-kserve-5c57bb7769-56bng"] Jul 05 12:32:46.129148 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:46.129116 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" path="/var/lib/kubelet/pods/3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b/volumes" Jul 05 12:32:46.129520 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:46.129507 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59eb9e58-ccec-4686-9e32-8d51b855c557" path="/var/lib/kubelet/pods/59eb9e58-ccec-4686-9e32-8d51b855c557/volumes" Jul 05 12:32:48.507168 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:48.507123 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:48.507570 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:48.507182 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:48.519472 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:48.519442 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:32:48.682686 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:32:48.682656 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:33:08.493438 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.493403 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt"] Jul 05 12:33:08.493916 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.493774 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59eb9e58-ccec-4686-9e32-8d51b855c557" containerName="main" Jul 05 12:33:08.493916 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.493786 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="59eb9e58-ccec-4686-9e32-8d51b855c557" containerName="main" Jul 05 12:33:08.493916 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.493808 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" containerName="main" Jul 05 12:33:08.493916 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.493813 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" containerName="main" Jul 05 12:33:08.493916 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.493904 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="59eb9e58-ccec-4686-9e32-8d51b855c557" containerName="main" Jul 05 12:33:08.493916 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.493915 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c14fcdc-14c4-43b3-8d4f-2b77e37aab9b" containerName="main" Jul 05 12:33:08.496918 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.496901 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.500917 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.500892 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"default-dockercfg-msbcg\"" Jul 05 12:33:08.501048 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.500917 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"openshift-service-ca.crt\"" Jul 05 12:33:08.501048 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.500934 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"kube-root-ca.crt\"" Jul 05 12:33:08.501048 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.500940 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-85c29d\"/\"rate-test-model-85c29d-kserve-self-signed-certs\"" Jul 05 12:33:08.504244 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.503957 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt"] Jul 05 12:33:08.565584 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.565546 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tmp-dir\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.565584 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.565595 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tls-certs\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.565831 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.565620 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-home\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.565831 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.565671 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-model-cache\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.565831 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.565775 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7p4d\" (UniqueName: \"kubernetes.io/projected/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-kube-api-access-j7p4d\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.565955 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.565856 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-dshm\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.584336 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.584303 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:08.667069 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.667023 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tls-certs\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.667069 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.667074 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-home\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.667324 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.667096 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-model-cache\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.667324 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.667124 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7p4d\" (UniqueName: \"kubernetes.io/projected/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-kube-api-access-j7p4d\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.667324 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.667163 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-dshm\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.667324 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.667221 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tmp-dir\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.667607 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.667571 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-model-cache\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.667717 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.667589 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tmp-dir\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.667717 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.667649 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-home\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.669552 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.669527 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-dshm\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.669831 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.669812 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tls-certs\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.676331 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.676301 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7p4d\" (UniqueName: \"kubernetes.io/projected/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-kube-api-access-j7p4d\") pod \"rate-test-model-85c29d-kserve-78ff744755-twcpt\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.807689 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.807653 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:08.949694 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:08.949664 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt"] Jul 05 12:33:08.952449 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:33:08.952423 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f0d37a8_64e0_4e26_b6fa_afc327e84d7e.slice/crio-4266af1e8ee47935aa3f8bf03a48e2b0c2d6990364353ca1cc04588ff8c222a9 WatchSource:0}: Error finding container 4266af1e8ee47935aa3f8bf03a48e2b0c2d6990364353ca1cc04588ff8c222a9: Status 404 returned error can't find the container with id 4266af1e8ee47935aa3f8bf03a48e2b0c2d6990364353ca1cc04588ff8c222a9 Jul 05 12:33:09.777823 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:09.777782 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" event={"ID":"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e","Type":"ContainerStarted","Data":"85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3"} Jul 05 12:33:09.778224 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:09.777833 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" event={"ID":"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e","Type":"ContainerStarted","Data":"4266af1e8ee47935aa3f8bf03a48e2b0c2d6990364353ca1cc04588ff8c222a9"} Jul 05 12:33:09.796050 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:09.795998 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" podStartSLOduration=1.79598334 podStartE2EDuration="1.79598334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:33:09.792849512 +0000 UTC m=+2390.183551275" watchObservedRunningTime="2026-07-05 12:33:09.79598334 +0000 UTC m=+2390.186685101" Jul 05 12:33:12.687691 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:12.687652 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:33:18.807767 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:18.807727 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:18.808186 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:18.807791 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:18.820947 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:18.820915 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:18.835063 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:18.835034 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:33:20.246380 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:20.246356 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:33:20.251167 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:20.251145 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:33:27.386890 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:27.386848 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:33:42.092620 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:42.092583 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:33:42.442161 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:42.442077 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:42.543144 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:42.543107 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:44.644727 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:44.644688 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:44.740728 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:44.740689 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:44.926072 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:44.925981 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:49.580677 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:49.580642 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p"] Jul 05 12:33:49.599214 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:49.599183 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:50.500734 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:50.500701 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt"] Jul 05 12:33:50.521775 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:50.521743 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:51.602930 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:51.602893 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:51.913084 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:51.913004 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:51.998169 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:51.998138 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:52.898640 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:52.898602 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:52.985812 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:52.985771 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:55.492765 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:55.492727 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:33:57.180986 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:57.180948 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:33:58.508088 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:33:58.508053 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:04.600899 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.600820 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" podUID="3158c734-327e-454d-bb5a-1c13f3d1d445" containerName="main" containerID="cri-o://51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da" gracePeriod=45 Jul 05 12:34:04.863204 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.863134 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:34:04.970156 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.970117 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-home\") pod \"3158c734-327e-454d-bb5a-1c13f3d1d445\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " Jul 05 12:34:04.970156 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.970159 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-model-cache\") pod \"3158c734-327e-454d-bb5a-1c13f3d1d445\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " Jul 05 12:34:04.970377 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.970193 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-dshm\") pod \"3158c734-327e-454d-bb5a-1c13f3d1d445\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " Jul 05 12:34:04.970377 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.970237 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3158c734-327e-454d-bb5a-1c13f3d1d445-tls-certs\") pod \"3158c734-327e-454d-bb5a-1c13f3d1d445\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " Jul 05 12:34:04.970377 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.970259 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-tmp-dir\") pod \"3158c734-327e-454d-bb5a-1c13f3d1d445\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " Jul 05 12:34:04.970377 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.970316 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gk7nw\" (UniqueName: \"kubernetes.io/projected/3158c734-327e-454d-bb5a-1c13f3d1d445-kube-api-access-gk7nw\") pod \"3158c734-327e-454d-bb5a-1c13f3d1d445\" (UID: \"3158c734-327e-454d-bb5a-1c13f3d1d445\") " Jul 05 12:34:04.970581 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.970487 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-model-cache" (OuterVolumeSpecName: "model-cache") pod "3158c734-327e-454d-bb5a-1c13f3d1d445" (UID: "3158c734-327e-454d-bb5a-1c13f3d1d445"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:34:04.970581 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.970522 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-home" (OuterVolumeSpecName: "home") pod "3158c734-327e-454d-bb5a-1c13f3d1d445" (UID: "3158c734-327e-454d-bb5a-1c13f3d1d445"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:34:04.970699 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.970574 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3158c734-327e-454d-bb5a-1c13f3d1d445" (UID: "3158c734-327e-454d-bb5a-1c13f3d1d445"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:34:04.972415 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.972382 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-dshm" (OuterVolumeSpecName: "dshm") pod "3158c734-327e-454d-bb5a-1c13f3d1d445" (UID: "3158c734-327e-454d-bb5a-1c13f3d1d445"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:34:04.972535 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.972409 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3158c734-327e-454d-bb5a-1c13f3d1d445-kube-api-access-gk7nw" (OuterVolumeSpecName: "kube-api-access-gk7nw") pod "3158c734-327e-454d-bb5a-1c13f3d1d445" (UID: "3158c734-327e-454d-bb5a-1c13f3d1d445"). InnerVolumeSpecName "kube-api-access-gk7nw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:34:04.972535 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:04.972455 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3158c734-327e-454d-bb5a-1c13f3d1d445-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3158c734-327e-454d-bb5a-1c13f3d1d445" (UID: "3158c734-327e-454d-bb5a-1c13f3d1d445"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:34:05.059076 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.059034 2575 generic.go:356] "Generic (PLEG): container finished" podID="3158c734-327e-454d-bb5a-1c13f3d1d445" containerID="51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da" exitCode=0 Jul 05 12:34:05.059076 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.059081 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" Jul 05 12:34:05.059279 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.059127 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" event={"ID":"3158c734-327e-454d-bb5a-1c13f3d1d445","Type":"ContainerDied","Data":"51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da"} Jul 05 12:34:05.059279 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.059175 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p" event={"ID":"3158c734-327e-454d-bb5a-1c13f3d1d445","Type":"ContainerDied","Data":"62ef5f8f8a040453f7fc36752f3da3cd56eb624befb06f8325c654ae8720ceb0"} Jul 05 12:34:05.059279 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.059195 2575 scope.go:117] "RemoveContainer" containerID="51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da" Jul 05 12:34:05.068307 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.068287 2575 scope.go:117] "RemoveContainer" containerID="51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da" Jul 05 12:34:05.068569 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:34:05.068551 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da\": container with ID starting with 51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da not found: ID does not exist" containerID="51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da" Jul 05 12:34:05.068629 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.068578 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da"} err="failed to get container status \"51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da\": rpc error: code = NotFound desc = could not find container \"51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da\": container with ID starting with 51845a8919d7254f5edb352aa75b72be58108f3e36a7dcd170bf49297d9d45da not found: ID does not exist" Jul 05 12:34:05.071138 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.071118 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-home\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:05.071236 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.071139 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-model-cache\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:05.071236 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.071152 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-dshm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:05.071236 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.071163 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3158c734-327e-454d-bb5a-1c13f3d1d445-tls-certs\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:05.071236 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.071174 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3158c734-327e-454d-bb5a-1c13f3d1d445-tmp-dir\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:05.071236 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.071186 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk7nw\" (UniqueName: \"kubernetes.io/projected/3158c734-327e-454d-bb5a-1c13f3d1d445-kube-api-access-gk7nw\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:05.080340 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.080314 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p"] Jul 05 12:34:05.082010 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.081991 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6f19c8/rate-test-model-6f19c8-kserve-58d5d777d7-h878p"] Jul 05 12:34:05.520770 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.520712 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" podUID="8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" containerName="main" containerID="cri-o://85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3" gracePeriod=45 Jul 05 12:34:05.821778 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.821746 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:34:05.999127 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.999088 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7p4d\" (UniqueName: \"kubernetes.io/projected/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-kube-api-access-j7p4d\") pod \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " Jul 05 12:34:05.999127 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.999135 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-model-cache\") pod \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " Jul 05 12:34:05.999403 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.999165 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tmp-dir\") pod \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " Jul 05 12:34:05.999403 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.999208 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-dshm\") pod \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " Jul 05 12:34:05.999403 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.999249 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tls-certs\") pod \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " Jul 05 12:34:05.999403 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.999289 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-home\") pod \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\" (UID: \"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e\") " Jul 05 12:34:05.999601 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.999434 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-model-cache" (OuterVolumeSpecName: "model-cache") pod "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" (UID: "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:34:05.999601 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.999485 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" (UID: "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:34:05.999601 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.999587 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-model-cache\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:05.999751 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.999606 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tmp-dir\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:05.999751 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:05.999640 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-home" (OuterVolumeSpecName: "home") pod "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" (UID: "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:34:06.001424 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.001400 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-dshm" (OuterVolumeSpecName: "dshm") pod "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" (UID: "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:34:06.001516 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.001451 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" (UID: "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:34:06.001571 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.001543 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-kube-api-access-j7p4d" (OuterVolumeSpecName: "kube-api-access-j7p4d") pod "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" (UID: "8f0d37a8-64e0-4e26-b6fa-afc327e84d7e"). InnerVolumeSpecName "kube-api-access-j7p4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:34:06.065993 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.065961 2575 generic.go:356] "Generic (PLEG): container finished" podID="8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" containerID="85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3" exitCode=0 Jul 05 12:34:06.066143 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.066002 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" Jul 05 12:34:06.066143 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.066054 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" event={"ID":"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e","Type":"ContainerDied","Data":"85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3"} Jul 05 12:34:06.066143 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.066101 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt" event={"ID":"8f0d37a8-64e0-4e26-b6fa-afc327e84d7e","Type":"ContainerDied","Data":"4266af1e8ee47935aa3f8bf03a48e2b0c2d6990364353ca1cc04588ff8c222a9"} Jul 05 12:34:06.066143 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.066118 2575 scope.go:117] "RemoveContainer" containerID="85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3" Jul 05 12:34:06.075390 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.075372 2575 scope.go:117] "RemoveContainer" containerID="85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3" Jul 05 12:34:06.075645 ip-10-0-142-207 kubenswrapper[2575]: E0705 12:34:06.075626 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3\": container with ID starting with 85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3 not found: ID does not exist" containerID="85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3" Jul 05 12:34:06.075690 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.075665 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3"} err="failed to get container status \"85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3\": rpc error: code = NotFound desc = could not find container \"85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3\": container with ID starting with 85157135ffb23e41f81c67a7cd8926cafb254e3837169a10e6515677930508b3 not found: ID does not exist" Jul 05 12:34:06.089594 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.089558 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt"] Jul 05 12:34:06.096350 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.096328 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-85c29d/rate-test-model-85c29d-kserve-78ff744755-twcpt"] Jul 05 12:34:06.100513 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.100495 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-tls-certs\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:06.100513 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.100514 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-home\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:06.100658 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.100524 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7p4d\" (UniqueName: \"kubernetes.io/projected/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-kube-api-access-j7p4d\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:06.100658 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.100534 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e-dshm\") on node \"ip-10-0-142-207.ec2.internal\" DevicePath \"\"" Jul 05 12:34:06.136075 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.136039 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3158c734-327e-454d-bb5a-1c13f3d1d445" path="/var/lib/kubelet/pods/3158c734-327e-454d-bb5a-1c13f3d1d445/volumes" Jul 05 12:34:06.136443 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:06.136431 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" path="/var/lib/kubelet/pods/8f0d37a8-64e0-4e26-b6fa-afc327e84d7e/volumes" Jul 05 12:34:31.819382 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:31.819333 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:32.140679 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:32.140587 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:32.385112 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:32.385068 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:32.561189 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:32.561144 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:32.876908 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:32.876812 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt"] Jul 05 12:34:32.878652 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:32.878624 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-85c29d-29720910-82tmt"] Jul 05 12:34:32.878784 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:32.878676 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:32.978429 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:32.978371 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:34.129930 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:34.129864 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c04386a-071a-4be3-b8a2-e2464ff95a72" path="/var/lib/kubelet/pods/5c04386a-071a-4be3-b8a2-e2464ff95a72/volumes" Jul 05 12:34:34.272716 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:34.272680 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:34.991143 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:34.991107 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c27fh"] Jul 05 12:34:38.731956 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:38.731921 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:38.800061 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:38.799997 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:39.015127 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:39.015027 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:39.245046 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:39.245007 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:44.621713 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:44.621678 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bg6zf_5cc93917-ad73-4c1e-83c6-0df9acb97352/manager/0.log" Jul 05 12:34:44.745720 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:44.745692 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mgw2t_b7d41648-4c6a-4218-98f9-ceb6b36ba9d3/manager/0.log" Jul 05 12:34:45.105436 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:45.105398 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720910-czldf_a0b48d6b-57e8-4411-8d77-6a50cdec700d/cleanup/0.log" Jul 05 12:34:45.354159 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:45.354133 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kc84j_441cc4e0-f699-4028-9cdb-bcf7d8128546/server/0.log" Jul 05 12:34:45.863492 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:45.863461 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59595c8d98-96cwh_8a1f1d34-d351-4872-a61d-861362025eac/manager/0.log" Jul 05 12:34:45.869011 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:45.868987 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59595c8d98-96cwh_8a1f1d34-d351-4872-a61d-861362025eac/copy-manifests/0.log" Jul 05 12:34:47.451521 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:47.451491 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tpw94_89bb08b7-fe0c-4536-b29d-a046f2534a65/manager/0.log" Jul 05 12:34:47.586112 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:47.586083 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hlp42_a22ce71e-6388-4407-adaa-0fc36fda26be/manager/0.log" Jul 05 12:34:47.820412 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:47.820381 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k4gb8_1d27ba7f-71e4-4e6c-b457-f9f4b461c4a7/registry-server/0.log" Jul 05 12:34:47.941532 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:47.941498 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-frmqx_bda3f19b-66ee-4a36-ab55-5ca30e2e6c46/manager/0.log" Jul 05 12:34:48.066854 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:48.066814 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-c27fh_eb38e1f8-7da8-4a0e-9416-5f70fbdcac50/limitador/0.log" Jul 05 12:34:48.189012 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:48.188931 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k77gs_bb0dab62-164f-4b62-9c10-14d76ede33ff/manager/0.log" Jul 05 12:34:48.916288 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:48.916255 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f99dd8ff-jf658_30705649-53ef-425c-940a-f0c8d125e5d4/kube-auth-proxy/0.log" Jul 05 12:34:49.389129 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:49.389104 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77bbc49f9c-hdjwj_842db65b-2a1c-4d56-bf76-72b3115a18f0/router/0.log" Jul 05 12:34:50.080155 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:50.080123 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7nbrs_ec4872c6-9576-4c62-9dac-a515bc97c92f/main/0.log" Jul 05 12:34:54.540326 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.540290 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dkr2v/must-gather-r24vt"] Jul 05 12:34:54.540805 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.540648 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3158c734-327e-454d-bb5a-1c13f3d1d445" containerName="main" Jul 05 12:34:54.540805 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.540659 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3158c734-327e-454d-bb5a-1c13f3d1d445" containerName="main" Jul 05 12:34:54.540805 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.540679 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" containerName="main" Jul 05 12:34:54.540805 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.540684 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" containerName="main" Jul 05 12:34:54.540805 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.540744 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3158c734-327e-454d-bb5a-1c13f3d1d445" containerName="main" Jul 05 12:34:54.540805 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.540754 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0d37a8-64e0-4e26-b6fa-afc327e84d7e" containerName="main" Jul 05 12:34:54.545239 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.545215 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dkr2v/must-gather-r24vt" Jul 05 12:34:54.547976 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.547953 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dkr2v\"/\"openshift-service-ca.crt\"" Jul 05 12:34:54.548098 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.548061 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dkr2v\"/\"default-dockercfg-9cplw\"" Jul 05 12:34:54.548154 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.548114 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dkr2v\"/\"kube-root-ca.crt\"" Jul 05 12:34:54.552351 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.552329 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dkr2v/must-gather-r24vt"] Jul 05 12:34:54.554138 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.554111 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:54.617143 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.617111 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" limit=1024 Jul 05 12:34:54.719307 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.719267 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/83c9a6d7-d375-44bb-93dd-cf2deaef71a9-must-gather-output\") pod \"must-gather-r24vt\" (UID: \"83c9a6d7-d375-44bb-93dd-cf2deaef71a9\") " pod="openshift-must-gather-dkr2v/must-gather-r24vt" Jul 05 12:34:54.719474 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.719402 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t424\" (UniqueName: \"kubernetes.io/projected/83c9a6d7-d375-44bb-93dd-cf2deaef71a9-kube-api-access-2t424\") pod \"must-gather-r24vt\" (UID: \"83c9a6d7-d375-44bb-93dd-cf2deaef71a9\") " pod="openshift-must-gather-dkr2v/must-gather-r24vt" Jul 05 12:34:54.820188 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.820152 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/83c9a6d7-d375-44bb-93dd-cf2deaef71a9-must-gather-output\") pod \"must-gather-r24vt\" (UID: \"83c9a6d7-d375-44bb-93dd-cf2deaef71a9\") " pod="openshift-must-gather-dkr2v/must-gather-r24vt" Jul 05 12:34:54.820346 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.820242 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t424\" (UniqueName: \"kubernetes.io/projected/83c9a6d7-d375-44bb-93dd-cf2deaef71a9-kube-api-access-2t424\") pod \"must-gather-r24vt\" (UID: \"83c9a6d7-d375-44bb-93dd-cf2deaef71a9\") " pod="openshift-must-gather-dkr2v/must-gather-r24vt" Jul 05 12:34:54.820508 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.820487 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/83c9a6d7-d375-44bb-93dd-cf2deaef71a9-must-gather-output\") pod \"must-gather-r24vt\" (UID: \"83c9a6d7-d375-44bb-93dd-cf2deaef71a9\") " pod="openshift-must-gather-dkr2v/must-gather-r24vt" Jul 05 12:34:54.828479 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.828450 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t424\" (UniqueName: \"kubernetes.io/projected/83c9a6d7-d375-44bb-93dd-cf2deaef71a9-kube-api-access-2t424\") pod \"must-gather-r24vt\" (UID: \"83c9a6d7-d375-44bb-93dd-cf2deaef71a9\") " pod="openshift-must-gather-dkr2v/must-gather-r24vt" Jul 05 12:34:54.855259 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.855230 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dkr2v/must-gather-r24vt" Jul 05 12:34:54.996358 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:54.996280 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dkr2v/must-gather-r24vt"] Jul 05 12:34:54.999776 ip-10-0-142-207 kubenswrapper[2575]: W0705 12:34:54.999741 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83c9a6d7_d375_44bb_93dd_cf2deaef71a9.slice/crio-9fe00a493bd46fbae7dc696ca16c97fc594398fc66ed17c99770052aa4cdd820 WatchSource:0}: Error finding container 9fe00a493bd46fbae7dc696ca16c97fc594398fc66ed17c99770052aa4cdd820: Status 404 returned error can't find the container with id 9fe00a493bd46fbae7dc696ca16c97fc594398fc66ed17c99770052aa4cdd820 Jul 05 12:34:55.301710 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:55.301562 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dkr2v/must-gather-r24vt" event={"ID":"83c9a6d7-d375-44bb-93dd-cf2deaef71a9","Type":"ContainerStarted","Data":"9fe00a493bd46fbae7dc696ca16c97fc594398fc66ed17c99770052aa4cdd820"} Jul 05 12:34:56.310009 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:56.309966 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dkr2v/must-gather-r24vt" event={"ID":"83c9a6d7-d375-44bb-93dd-cf2deaef71a9","Type":"ContainerStarted","Data":"88987c2bfa7d55fbe18458cfff3f59376bddb1bb1504c5cb1f0154b0a834804a"} Jul 05 12:34:56.310009 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:56.310015 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dkr2v/must-gather-r24vt" event={"ID":"83c9a6d7-d375-44bb-93dd-cf2deaef71a9","Type":"ContainerStarted","Data":"e62634bf37bf7b45d6cb1255be63f4b8f55c6776d6cf2181f92ddcfa36254bb7"} Jul 05 12:34:56.325009 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:56.324951 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dkr2v/must-gather-r24vt" podStartSLOduration=1.551635472 podStartE2EDuration="2.324936885s" totalImagesPullingTime="773.301413ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:34:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:34:56.324460981 +0000 UTC m=+2496.715162757" watchObservedRunningTime="2026-07-05 12:34:56.324936885 +0000 UTC m=+2496.715638646" Jul 05 12:34:57.348793 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:57.348764 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8z5fg_746ca75a-b87d-4153-9e6a-74845453758c/global-pull-secret-syncer/0.log" Jul 05 12:34:57.478496 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:57.478453 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j6nrt_d473ee62-6227-440a-bad0-ef141bb75628/konnectivity-agent/0.log" Jul 05 12:34:57.582070 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:34:57.582038 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-207.ec2.internal_3494324bd81becfa329adf53f8dc35af/haproxy/0.log" Jul 05 12:35:02.699370 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:02.699332 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tpw94_89bb08b7-fe0c-4536-b29d-a046f2534a65/manager/0.log" Jul 05 12:35:02.728046 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:02.728011 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hlp42_a22ce71e-6388-4407-adaa-0fc36fda26be/manager/0.log" Jul 05 12:35:02.794014 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:02.793983 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k4gb8_1d27ba7f-71e4-4e6c-b457-f9f4b461c4a7/registry-server/0.log" Jul 05 12:35:02.884795 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:02.884765 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-frmqx_bda3f19b-66ee-4a36-ab55-5ca30e2e6c46/manager/0.log" Jul 05 12:35:02.910023 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:02.909926 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-c27fh_eb38e1f8-7da8-4a0e-9416-5f70fbdcac50/limitador/0.log" Jul 05 12:35:02.994658 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:02.994428 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k77gs_bb0dab62-164f-4b62-9c10-14d76ede33ff/manager/0.log" Jul 05 12:35:04.973302 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:04.973266 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-ckggg_e13f32f9-100f-456f-bb39-1dac54b64af9/cluster-monitoring-operator/0.log" Jul 05 12:35:05.235754 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:05.235673 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7d6dg_57363bc0-e79d-4f28-b2e9-35a3a9c63b1a/node-exporter/0.log" Jul 05 12:35:05.259689 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:05.259660 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7d6dg_57363bc0-e79d-4f28-b2e9-35a3a9c63b1a/kube-rbac-proxy/0.log" Jul 05 12:35:05.281120 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:05.280949 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7d6dg_57363bc0-e79d-4f28-b2e9-35a3a9c63b1a/init-textfile/0.log" Jul 05 12:35:07.121987 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:07.121957 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-8pfwm_c843d931-14bd-431f-9263-9671ea3401a8/networking-console-plugin/0.log" Jul 05 12:35:07.686927 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:07.683487 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/1.log" Jul 05 12:35:07.689426 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:07.689397 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4sth7_5f7f19ec-9537-4581-b1ad-62d17debda03/console-operator/2.log" Jul 05 12:35:07.836250 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:07.836215 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm"] Jul 05 12:35:07.844613 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:07.844579 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:07.848577 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:07.848540 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm"] Jul 05 12:35:07.952219 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:07.952134 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-sys\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:07.952474 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:07.952453 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-lib-modules\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:07.952668 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:07.952649 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-proc\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:07.952777 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:07.952764 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-podres\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:07.952922 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:07.952905 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll9zl\" (UniqueName: \"kubernetes.io/projected/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-kube-api-access-ll9zl\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.054546 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.054503 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll9zl\" (UniqueName: \"kubernetes.io/projected/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-kube-api-access-ll9zl\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.054738 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.054578 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-sys\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.054738 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.054604 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-lib-modules\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.054738 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.054660 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-sys\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.054738 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.054668 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-proc\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.054738 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.054723 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-podres\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.055068 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.054750 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-proc\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.055068 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.054805 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-lib-modules\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.055068 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.054905 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-podres\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.066515 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.066485 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll9zl\" (UniqueName: \"kubernetes.io/projected/d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778-kube-api-access-ll9zl\") pod \"perf-node-gather-daemonset-9rswm\" (UID: \"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.158130 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.158093 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:08.362977 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.361260 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm"] Jul 05 12:35:08.404108 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.401364 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" event={"ID":"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778","Type":"ContainerStarted","Data":"e95f7ed740aaef30a6f26354025f625d860d0fb08ba3d5a8ddc06834df1427fe"} Jul 05 12:35:08.913914 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:08.913855 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-kh5kf_f3da3ff4-6df1-4407-8fd1-02ef70c431b6/volume-data-source-validator/0.log" Jul 05 12:35:09.411389 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:09.411339 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" event={"ID":"d3ce8dd1-fbd7-4ab1-986c-c4fccc0e1778","Type":"ContainerStarted","Data":"31ac6abb077ef43a2f14a79ea362f6055666fdc3798b4183a08983d659324691"} Jul 05 12:35:09.412335 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:09.412304 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:09.429370 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:09.429312 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" podStartSLOduration=2.429293232 podStartE2EDuration="2.429293232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 12:35:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:35:09.428174585 +0000 UTC m=+2509.818876349" watchObservedRunningTime="2026-07-05 12:35:09.429293232 +0000 UTC m=+2509.819994994" Jul 05 12:35:09.905583 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:09.905554 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gplkw_31952da1-f50a-4817-b342-775273f214f5/dns/0.log" Jul 05 12:35:09.927661 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:09.927636 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gplkw_31952da1-f50a-4817-b342-775273f214f5/kube-rbac-proxy/0.log" Jul 05 12:35:10.098550 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:10.098519 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hb68f_9930e6e9-d5bc-4384-a9de-97740b2b692d/dns-node-resolver/0.log" Jul 05 12:35:10.676754 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:10.676725 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s2dpw_afa4b888-1bc8-4865-911e-5720c5b0028a/node-ca/0.log" Jul 05 12:35:11.779169 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:11.779130 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f99dd8ff-jf658_30705649-53ef-425c-940a-f0c8d125e5d4/kube-auth-proxy/0.log" Jul 05 12:35:11.967906 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:11.967853 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77bbc49f9c-hdjwj_842db65b-2a1c-4d56-bf76-72b3115a18f0/router/0.log" Jul 05 12:35:12.628087 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:12.627986 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f67t2_4b444ec4-9b23-422d-ac9d-dd3535492d17/serve-healthcheck-canary/0.log" Jul 05 12:35:13.101254 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:13.101220 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-kf6hq_7197549e-9314-42dd-bbdb-d53feb6ece0e/insights-operator/0.log" Jul 05 12:35:13.101991 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:13.101968 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-kf6hq_7197549e-9314-42dd-bbdb-d53feb6ece0e/insights-operator/1.log" Jul 05 12:35:13.246374 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:13.246343 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m7tqb_42dcc512-1104-4580-a06a-42db9e8317f7/kube-rbac-proxy/0.log" Jul 05 12:35:13.266899 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:13.266850 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m7tqb_42dcc512-1104-4580-a06a-42db9e8317f7/exporter/0.log" Jul 05 12:35:13.288679 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:13.288650 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m7tqb_42dcc512-1104-4580-a06a-42db9e8317f7/extractor/0.log" Jul 05 12:35:15.372061 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:15.372031 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bg6zf_5cc93917-ad73-4c1e-83c6-0df9acb97352/manager/0.log" Jul 05 12:35:15.526417 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:15.526383 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mgw2t_b7d41648-4c6a-4218-98f9-ceb6b36ba9d3/manager/0.log" Jul 05 12:35:15.602376 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:15.602351 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720910-czldf_a0b48d6b-57e8-4411-8d77-6a50cdec700d/cleanup/0.log" Jul 05 12:35:15.714081 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:15.714005 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kc84j_441cc4e0-f699-4028-9cdb-bcf7d8128546/server/0.log" Jul 05 12:35:15.931688 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:15.931640 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59595c8d98-96cwh_8a1f1d34-d351-4872-a61d-861362025eac/manager/0.log" Jul 05 12:35:15.951951 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:15.951924 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59595c8d98-96cwh_8a1f1d34-d351-4872-a61d-861362025eac/copy-manifests/0.log" Jul 05 12:35:16.435522 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:16.434813 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-9rswm" Jul 05 12:35:18.046496 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:18.046470 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f99dd8ff-jf658_30705649-53ef-425c-940a-f0c8d125e5d4/kube-auth-proxy/0.log" Jul 05 12:35:18.169767 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:18.169720 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77bbc49f9c-hdjwj_842db65b-2a1c-4d56-bf76-72b3115a18f0/router/0.log" Jul 05 12:35:19.591518 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:19.591485 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f99dd8ff-jf658_30705649-53ef-425c-940a-f0c8d125e5d4/kube-auth-proxy/0.log" Jul 05 12:35:19.719705 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:19.719677 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77bbc49f9c-hdjwj_842db65b-2a1c-4d56-bf76-72b3115a18f0/router/0.log" Jul 05 12:35:22.477637 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:22.477607 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kt5dh_87856c3e-d6be-483e-b830-7b13000c87c5/migrator/0.log" Jul 05 12:35:22.496491 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:22.496466 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kt5dh_87856c3e-d6be-483e-b830-7b13000c87c5/graceful-termination/0.log" Jul 05 12:35:22.843144 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:22.843112 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-j8f2q_be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34/kube-storage-version-migrator-operator/1.log" Jul 05 12:35:22.844488 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:22.844453 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-j8f2q_be6fbe1d-9c8a-4e98-bc0a-f244bd1bad34/kube-storage-version-migrator-operator/0.log" Jul 05 12:35:23.809920 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:23.809867 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bc4j_43726046-5603-4270-94d8-4e3f27f984b7/kube-multus-additional-cni-plugins/0.log" Jul 05 12:35:23.833283 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:23.833253 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bc4j_43726046-5603-4270-94d8-4e3f27f984b7/egress-router-binary-copy/0.log" Jul 05 12:35:23.853832 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:23.853801 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bc4j_43726046-5603-4270-94d8-4e3f27f984b7/cni-plugins/0.log" Jul 05 12:35:23.874363 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:23.874335 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bc4j_43726046-5603-4270-94d8-4e3f27f984b7/bond-cni-plugin/0.log" Jul 05 12:35:23.894035 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:23.894007 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bc4j_43726046-5603-4270-94d8-4e3f27f984b7/routeoverride-cni/0.log" Jul 05 12:35:23.913351 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:23.913326 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bc4j_43726046-5603-4270-94d8-4e3f27f984b7/whereabouts-cni-bincopy/0.log" Jul 05 12:35:23.933416 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:23.933392 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6bc4j_43726046-5603-4270-94d8-4e3f27f984b7/whereabouts-cni/0.log" Jul 05 12:35:24.263716 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:24.263640 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gjqwd_9163d99b-81f1-49d9-85c4-e35200fb1b1c/kube-multus/0.log" Jul 05 12:35:24.419767 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:24.419732 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dhzh9_793d5b93-3b3a-4f7b-99fb-38db6730bfa6/network-metrics-daemon/0.log" Jul 05 12:35:24.437553 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:24.437526 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dhzh9_793d5b93-3b3a-4f7b-99fb-38db6730bfa6/kube-rbac-proxy/0.log" Jul 05 12:35:25.454483 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:25.454456 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngxcf_447db484-5c4b-4e4e-86f7-acbcb5c4f9cf/ovn-controller/0.log" Jul 05 12:35:25.469738 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:25.469711 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngxcf_447db484-5c4b-4e4e-86f7-acbcb5c4f9cf/ovn-acl-logging/0.log" Jul 05 12:35:25.490397 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:25.490360 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngxcf_447db484-5c4b-4e4e-86f7-acbcb5c4f9cf/kube-rbac-proxy-node/0.log" Jul 05 12:35:25.511262 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:25.511239 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngxcf_447db484-5c4b-4e4e-86f7-acbcb5c4f9cf/kube-rbac-proxy-ovn-metrics/0.log" Jul 05 12:35:25.528450 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:25.528414 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngxcf_447db484-5c4b-4e4e-86f7-acbcb5c4f9cf/northd/0.log" Jul 05 12:35:25.548358 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:25.548335 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngxcf_447db484-5c4b-4e4e-86f7-acbcb5c4f9cf/nbdb/0.log" Jul 05 12:35:25.567502 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:25.567477 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngxcf_447db484-5c4b-4e4e-86f7-acbcb5c4f9cf/sbdb/0.log" Jul 05 12:35:25.620308 ip-10-0-142-207 kubenswrapper[2575]: I0705 12:35:25.620272 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngxcf_447db484-5c4b-4e4e-86f7-acbcb5c4f9cf/ovnkube-controller/0.log"