Jul 07 14:33:14.610130 ip-10-0-143-146 systemd[1]: Starting Kubernetes Kubelet... Jul 07 14:33:15.016517 ip-10-0-143-146 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 14:33:15.016517 ip-10-0-143-146 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 14:33:15.016517 ip-10-0-143-146 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 14:33:15.016517 ip-10-0-143-146 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 14:33:15.016517 ip-10-0-143-146 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 14:33:15.017549 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.017466 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 14:33:15.021919 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021904 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:33:15.021919 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021919 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021923 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021926 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021930 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021933 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021937 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021940 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021943 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021946 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021949 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021951 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021962 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021965 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021968 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021970 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021973 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021975 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021978 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021980 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021983 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:33:15.021984 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021985 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021988 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021991 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021994 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.021997 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022000 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022002 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022005 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022008 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022010 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022013 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022016 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022018 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022020 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022023 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022026 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022028 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022031 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022033 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022036 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:33:15.022471 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022039 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022041 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022044 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022046 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022049 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022052 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022054 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022057 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022059 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022062 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022064 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022067 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022069 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022072 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022075 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022077 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022080 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022082 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022085 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022087 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:33:15.022952 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022090 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022092 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022095 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022098 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022101 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022103 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022106 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022110 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022113 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022116 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022120 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022124 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022127 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022132 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022135 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022138 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022140 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022143 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022146 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:33:15.023430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022148 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022151 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022153 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022156 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022158 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022161 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022163 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022167 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022170 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022172 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022175 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022177 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022180 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022183 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022186 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022189 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022192 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022197 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.022200 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:33:15.023882 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023746 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023753 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023756 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023759 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023762 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023765 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023767 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023770 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023772 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023775 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023777 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023780 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023783 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023786 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023788 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023791 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023793 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023796 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023799 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:33:15.024332 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023801 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023804 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023807 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023809 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023812 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023829 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023832 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023835 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023837 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023840 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023843 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023845 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023849 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023851 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023854 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023857 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023861 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023865 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023868 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023872 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:33:15.024826 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023875 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023877 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023881 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023884 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023887 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023890 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023892 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023895 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023898 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023900 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023903 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023905 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023908 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023911 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023914 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023917 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023920 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023922 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023925 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023928 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:33:15.025295 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023930 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023933 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023935 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023938 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023940 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023944 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023946 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023949 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023951 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023956 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023959 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023962 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023964 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023967 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023970 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023974 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023977 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023981 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023985 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:33:15.025783 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023988 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023991 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023993 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.023997 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024000 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024003 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024005 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024008 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024011 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024013 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024016 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024018 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024021 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024023 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024026 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024028 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024031 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024033 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024036 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024042 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:33:15.026266 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024045 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024427 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024432 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024435 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024438 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024440 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024443 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024445 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024448 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024450 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024453 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024455 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024458 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024460 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024463 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024465 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024468 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024470 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024473 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024476 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:33:15.026842 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024478 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024481 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024483 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024486 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024489 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024491 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024494 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024496 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024499 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024501 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024503 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024506 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024509 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024511 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024514 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024516 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024519 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024521 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024524 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024526 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:33:15.027324 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024528 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024531 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024534 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024536 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024539 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024542 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024545 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024547 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024550 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024552 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024555 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024558 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024563 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024566 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024568 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024570 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024573 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024575 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024578 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:33:15.027801 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024580 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024583 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024585 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024588 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024590 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024593 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024596 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024600 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024603 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024607 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024609 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024612 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024615 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024617 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024620 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024623 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024625 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024628 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024630 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024633 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:33:15.028270 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024635 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024640 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024643 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024647 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024649 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024652 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024655 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024657 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024660 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024662 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024665 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024668 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024672 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024675 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024677 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024680 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024682 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024685 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024688 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:33:15.028732 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024691 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.024693 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024763 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024771 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024776 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024781 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024785 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024788 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024792 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024796 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024799 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024802 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024806 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024809 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024812 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024827 2581 flags.go:64] FLAG: --cgroup-root="" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024831 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024834 2581 flags.go:64] FLAG: --client-ca-file="" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024837 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024841 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024845 2581 flags.go:64] FLAG: --cluster-domain="" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024848 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024851 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024857 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 14:33:15.029205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024860 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024864 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024867 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024870 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024873 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024877 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024880 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024883 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024886 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024889 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024893 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024896 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024899 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024902 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024905 2581 flags.go:64] FLAG: --enable-server="true" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024908 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024912 2581 flags.go:64] FLAG: --event-burst="100" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024915 2581 flags.go:64] FLAG: --event-qps="50" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024918 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024921 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024924 2581 flags.go:64] FLAG: --eviction-hard="" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024928 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024932 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024935 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024938 2581 flags.go:64] FLAG: --eviction-soft="" Jul 07 14:33:15.029774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024942 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024944 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024947 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024950 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024953 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024956 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024961 2581 flags.go:64] FLAG: --feature-gates="" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024965 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024967 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024970 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024973 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024977 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024980 2581 flags.go:64] FLAG: --help="false" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024982 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024986 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024989 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024992 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024995 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.024999 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025001 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025004 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025007 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025010 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 14:33:15.030412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025013 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025016 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025019 2581 flags.go:64] FLAG: --kube-reserved="" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025022 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025024 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025027 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025032 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025035 2581 flags.go:64] FLAG: --lock-file="" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025038 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025041 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025044 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025049 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025052 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025054 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025057 2581 flags.go:64] FLAG: --logging-format="text" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025061 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025065 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025068 2581 flags.go:64] FLAG: --manifest-url="" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025071 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025075 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025078 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025082 2581 flags.go:64] FLAG: --max-pods="110" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025085 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025088 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025091 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 14:33:15.031001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025094 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025097 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025100 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025103 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025111 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025114 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025117 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025119 2581 flags.go:64] FLAG: --pod-cidr="" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025122 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025127 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025130 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025134 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025137 2581 flags.go:64] FLAG: --port="10250" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025139 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025144 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09fece66530bcd81f" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025148 2581 flags.go:64] FLAG: --qos-reserved="" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025151 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025154 2581 flags.go:64] FLAG: --register-node="true" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025157 2581 flags.go:64] FLAG: --register-with-taints="" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025161 2581 flags.go:64] FLAG: --registry-burst="10" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025164 2581 flags.go:64] FLAG: --registry-qps="5" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025167 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025171 2581 flags.go:64] FLAG: --reserved-memory="" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025175 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025178 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 14:33:15.031602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025181 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025183 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025186 2581 flags.go:64] FLAG: --runonce="false" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025189 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025192 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025195 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025198 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025201 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025204 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025207 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025210 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025213 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025216 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025219 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025221 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025224 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025227 2581 flags.go:64] FLAG: --system-cgroups="" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025229 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025235 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025237 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025240 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025246 2581 flags.go:64] FLAG: --tls-min-version="" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025248 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025251 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025254 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 14:33:15.032240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025257 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025260 2581 flags.go:64] FLAG: --v="2" Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025264 2581 flags.go:64] FLAG: --version="false" Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025268 2581 flags.go:64] FLAG: --vmodule="" Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025273 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.025277 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025393 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025397 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025399 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025402 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025405 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025407 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025410 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025412 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025414 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025417 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025419 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025423 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025425 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025427 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025430 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:33:15.032850 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025433 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025436 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025438 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025441 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025443 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025446 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025448 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025451 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025454 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025456 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025459 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025461 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025464 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025466 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025469 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025472 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025475 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025478 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025480 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:33:15.033349 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025482 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025485 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025488 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025490 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025493 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025495 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025499 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025503 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025505 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025508 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025511 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025513 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025516 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025518 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025520 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025523 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025526 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025528 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025530 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025533 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:33:15.033854 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025535 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025538 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025541 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025543 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025545 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025548 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025550 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025553 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025556 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025559 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025561 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025564 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025566 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025569 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025571 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025573 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025576 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025578 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025580 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025583 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:33:15.034397 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025585 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025588 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025590 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025593 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025595 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025598 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025600 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025602 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025605 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025607 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025610 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025612 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025615 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025618 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025622 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025625 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025627 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025630 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025632 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025635 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:33:15.035181 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025638 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025641 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025643 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025647 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.025650 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.026478 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.033548 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.033563 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.034111 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.034121 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034179 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034185 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034189 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034192 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:33:15.035720 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034195 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034238 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034277 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034280 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034283 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034286 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034289 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034382 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034388 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034391 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034396 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034400 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034406 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034410 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034420 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034424 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034428 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034432 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034436 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034440 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:33:15.036098 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034444 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034448 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034452 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034456 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034460 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034465 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034470 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034478 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034482 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034486 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034493 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034498 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034502 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034506 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034511 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034516 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034520 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034524 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034528 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:33:15.036582 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034537 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034541 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034545 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034549 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034553 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034557 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034562 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034566 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034571 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034575 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034581 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034587 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034596 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034602 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034608 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034613 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034618 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034622 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034627 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:33:15.037143 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034632 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034636 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034641 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034646 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034650 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034658 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034669 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034676 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034680 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034686 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034692 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034697 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034701 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034705 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034710 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034714 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034718 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034722 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034731 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:33:15.037594 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034736 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034740 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034744 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034748 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034753 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034757 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034761 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034765 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034770 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034774 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034778 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034782 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034792 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034796 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034800 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034804 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034808 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:33:15.038069 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.034812 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.034836 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035205 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035212 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035218 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035225 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035230 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035234 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035239 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035243 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035247 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035251 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035261 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035265 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035270 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:33:15.038545 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035274 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035278 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035283 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035287 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035291 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035295 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035300 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035304 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035308 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035318 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035322 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035326 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035331 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035336 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035339 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035342 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035345 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035351 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035353 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035356 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:33:15.038927 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035359 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035361 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035364 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035367 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035371 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035373 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035376 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035381 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035385 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035389 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035408 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035426 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035429 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035519 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035530 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035535 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035538 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035541 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035544 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035549 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:33:15.039398 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035551 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035554 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035557 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035560 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035562 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035565 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035568 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035570 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035575 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035579 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035582 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035585 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035588 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035591 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035593 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035596 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035598 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035604 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035607 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:33:15.039954 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035610 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035613 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035615 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035618 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035621 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035623 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035626 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035629 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035632 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035635 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035638 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035640 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035643 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035646 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035649 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035652 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035654 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035657 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035660 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035662 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:33:15.040406 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035665 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035667 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035670 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035673 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035676 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035679 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:15.035683 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.035689 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.035842 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.038401 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.039774 2581 server.go:1019] "Starting client certificate rotation" Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.039851 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 14:33:15.040892 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.039883 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 14:33:15.042320 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.042307 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 14:33:15.045346 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.045320 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 14:33:15.054147 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.054132 2581 log.go:25] "Validated CRI v1 runtime API" Jul 07 14:33:15.056171 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.056154 2581 log.go:25] "Validated CRI v1 image API" Jul 07 14:33:15.056961 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.056949 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 14:33:15.060213 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.060196 2581 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a035f99d-b6df-48d4-99cc-c037eebc9fd5:/dev/nvme0n1p3 db82bb4e-8afb-4a70-9008-bd5bb587562d:/dev/nvme0n1p4] Jul 07 14:33:15.060255 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.060214 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 14:33:15.066611 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.066484 2581 manager.go:216] Machine: {Timestamp:2026-07-07 14:33:15.06515878 +0000 UTC m=+0.340242218 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095322 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c615a2b340ad927dbf60a69dae1c0 SystemUUID:ec2c615a-2b34-0ad9-27db-f60a69dae1c0 BootID:4ff55be9-a474-465e-98e3-344d107ad0b3 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:76:13:41:6f:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:76:13:41:6f:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:42:5c:66:55:9e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 14:33:15.066611 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.066609 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 14:33:15.066738 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.066727 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 14:33:15.069163 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.069136 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 14:33:15.069304 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.069171 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-146.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 14:33:15.069844 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.069834 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 14:33:15.069877 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.069847 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 14:33:15.069877 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.069854 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 14:33:15.070489 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.070480 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 14:33:15.071717 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.071708 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 14:33:15.074599 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.074588 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 07 14:33:15.074728 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.074719 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 14:33:15.075419 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.075410 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 07 14:33:15.075450 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.075439 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 14:33:15.075483 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.075454 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 14:33:15.075520 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.075513 2581 kubelet.go:398] "Adding apiserver pod source" Jul 07 14:33:15.075549 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.075524 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 14:33:15.076690 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.076676 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 14:33:15.076750 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.076704 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 14:33:15.078376 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.078350 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 14:33:15.080582 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.080562 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 14:33:15.082656 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.082641 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 14:33:15.082726 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.082672 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 14:33:15.083880 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083868 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 14:33:15.083921 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083885 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 14:33:15.083921 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083892 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 14:33:15.083921 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083899 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 14:33:15.083921 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083911 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 14:33:15.083921 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083920 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 14:33:15.084053 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083931 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 14:33:15.084053 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083937 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 14:33:15.084053 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083944 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 14:33:15.084053 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083950 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 14:33:15.084053 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083959 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 14:33:15.084053 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.083964 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 14:33:15.084689 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.084680 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 14:33:15.084689 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.084689 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 14:33:15.087957 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.087942 2581 server.go:1268] "Started kubelet" Jul 07 14:33:15.088038 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.088001 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 14:33:15.088100 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.088041 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 14:33:15.088139 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.088127 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 14:33:15.088563 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.088542 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-146.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 14:33:15.088655 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.088537 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 14:33:15.088655 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.088581 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 14:33:15.088792 ip-10-0-143-146 systemd[1]: Started Kubernetes Kubelet. Jul 07 14:33:15.089919 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.089903 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 14:33:15.090491 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.090477 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 07 14:33:15.094294 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.094273 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 14:33:15.097083 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.097067 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 14:33:15.097083 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.097076 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 14:33:15.097202 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.096150 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-146.ec2.internal.18c0087f414954e9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-146.ec2.internal,UID:ip-10-0-143-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-146.ec2.internal,},FirstTimestamp:2026-07-07 14:33:15.087910121 +0000 UTC m=+0.362993538,LastTimestamp:2026-07-07 14:33:15.087910121 +0000 UTC m=+0.362993538,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-146.ec2.internal,}" Jul 07 14:33:15.097256 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.097220 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:15.097721 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.097693 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 14:33:15.097721 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.097717 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 14:33:15.097848 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.097692 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 14:33:15.097848 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.097832 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 07 14:33:15.097848 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.097843 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 07 14:33:15.098071 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.098050 2581 factory.go:55] Registering systemd factory Jul 07 14:33:15.098071 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.098075 2581 factory.go:223] Registration of the systemd container factory successfully Jul 07 14:33:15.098370 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.098357 2581 factory.go:153] Registering CRI-O factory Jul 07 14:33:15.098407 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.098374 2581 factory.go:223] Registration of the crio container factory successfully Jul 07 14:33:15.098447 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.098436 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 14:33:15.098478 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.098462 2581 factory.go:103] Registering Raw factory Jul 07 14:33:15.098515 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.098487 2581 manager.go:1196] Started watching for new ooms in manager Jul 07 14:33:15.098968 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.098952 2581 manager.go:349] Starting recovery of all containers Jul 07 14:33:15.105603 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.105557 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 14:33:15.105724 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.105664 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 14:33:15.112338 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.112316 2581 manager.go:354] Recovery completed Jul 07 14:33:15.116519 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.116501 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:33:15.118708 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.118688 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:33:15.118773 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.118724 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:33:15.118773 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.118738 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:33:15.119330 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.119316 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 14:33:15.119330 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.119330 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 14:33:15.119422 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.119353 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 07 14:33:15.119422 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.119332 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:33:15.120165 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.120150 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:33:15.120220 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.120180 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:33:15.120220 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.120191 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:33:15.120598 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.120585 2581 policy_none.go:49] "None policy: Start" Jul 07 14:33:15.120648 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.120604 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 14:33:15.120648 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.120619 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 14:33:15.121059 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.120984 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-146.ec2.internal.18c0087f431f4909 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-146.ec2.internal,UID:ip-10-0-143-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-146.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-146.ec2.internal,},FirstTimestamp:2026-07-07 14:33:15.118709001 +0000 UTC m=+0.393792423,LastTimestamp:2026-07-07 14:33:15.118709001 +0000 UTC m=+0.393792423,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-146.ec2.internal,}" Jul 07 14:33:15.121679 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.121668 2581 policy_none.go:47] "Start" Jul 07 14:33:15.131847 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.131658 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-146.ec2.internal.18c0087f431f9f8f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-146.ec2.internal,UID:ip-10-0-143-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-146.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-146.ec2.internal,},FirstTimestamp:2026-07-07 14:33:15.118731151 +0000 UTC m=+0.393814570,LastTimestamp:2026-07-07 14:33:15.118731151 +0000 UTC m=+0.393814570,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-146.ec2.internal,}" Jul 07 14:33:15.134458 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.134440 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jw5hh" Jul 07 14:33:15.142291 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.142270 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jw5hh" Jul 07 14:33:15.142488 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.142401 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-146.ec2.internal.18c0087f431fd08e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-146.ec2.internal,UID:ip-10-0-143-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-146.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-146.ec2.internal,},FirstTimestamp:2026-07-07 14:33:15.118743694 +0000 UTC m=+0.393827112,LastTimestamp:2026-07-07 14:33:15.118743694 +0000 UTC m=+0.393827112,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-146.ec2.internal,}" Jul 07 14:33:15.198060 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.198032 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:15.234392 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.234351 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 14:33:15.235539 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.235522 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 14:33:15.235652 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.235565 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 14:33:15.235652 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.235602 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 14:33:15.235652 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.235642 2581 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 07 14:33:15.238125 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.238106 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:33:15.298194 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.298108 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:15.306390 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.306375 2581 manager.go:337] "Starting Device Plugin manager" Jul 07 14:33:15.306473 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.306408 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 14:33:15.306473 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.306418 2581 server.go:87] "Starting device plugin registration server" Jul 07 14:33:15.306683 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.306670 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 14:33:15.306732 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.306701 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 14:33:15.306873 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.306847 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 14:33:15.307031 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.307015 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 14:33:15.307031 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.307029 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 14:33:15.307561 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.307540 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 14:33:15.307652 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.307590 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:15.313519 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.313500 2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-146.ec2.internal\" not found" node="ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.336742 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.336704 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-146.ec2.internal"] Jul 07 14:33:15.336851 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.336783 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:33:15.337761 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.337746 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:33:15.337844 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.337777 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:33:15.337844 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.337789 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:33:15.339107 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.339095 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:33:15.339221 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.339207 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.339261 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.339237 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:33:15.340781 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.340764 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:33:15.340781 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.340778 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:33:15.340889 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.340790 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:33:15.340889 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.340796 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:33:15.340889 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.340803 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:33:15.340889 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.340831 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:33:15.342164 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.342145 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.342164 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.342167 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:33:15.342907 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.342892 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:33:15.342984 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.342916 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:33:15.342984 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.342925 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:33:15.366420 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.366394 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-146.ec2.internal\" not found" node="ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.370697 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.370683 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-146.ec2.internal\" not found" node="ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.407431 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.407409 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:33:15.408427 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.408413 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:33:15.408500 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.408439 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:33:15.408500 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.408450 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:33:15.408569 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.408508 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.416706 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.416692 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.416767 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.416713 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-146.ec2.internal\": node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:15.438470 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.438445 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:15.499240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.499198 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d9df984a94f344c9e5dde59a74f6e184-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal\" (UID: \"d9df984a94f344c9e5dde59a74f6e184\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.499240 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.499241 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9df984a94f344c9e5dde59a74f6e184-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal\" (UID: \"d9df984a94f344c9e5dde59a74f6e184\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.499372 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.499260 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/00c897cdd111a3fcc4bf6f54f3d25813-config\") pod \"kube-apiserver-proxy-ip-10-0-143-146.ec2.internal\" (UID: \"00c897cdd111a3fcc4bf6f54f3d25813\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.539360 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.539292 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:15.599931 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.599891 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d9df984a94f344c9e5dde59a74f6e184-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal\" (UID: \"d9df984a94f344c9e5dde59a74f6e184\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.599997 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.599936 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9df984a94f344c9e5dde59a74f6e184-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal\" (UID: \"d9df984a94f344c9e5dde59a74f6e184\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.599997 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.599968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/00c897cdd111a3fcc4bf6f54f3d25813-config\") pod \"kube-apiserver-proxy-ip-10-0-143-146.ec2.internal\" (UID: \"00c897cdd111a3fcc4bf6f54f3d25813\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.600061 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.600008 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d9df984a94f344c9e5dde59a74f6e184-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal\" (UID: \"d9df984a94f344c9e5dde59a74f6e184\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.600061 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.600013 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9df984a94f344c9e5dde59a74f6e184-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal\" (UID: \"d9df984a94f344c9e5dde59a74f6e184\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.600061 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.600037 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/00c897cdd111a3fcc4bf6f54f3d25813-config\") pod \"kube-apiserver-proxy-ip-10-0-143-146.ec2.internal\" (UID: \"00c897cdd111a3fcc4bf6f54f3d25813\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.640050 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.640020 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:15.670199 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.670178 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.674376 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.673965 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-146.ec2.internal" Jul 07 14:33:15.741096 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.741065 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:15.841543 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.841518 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:15.942517 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:15.942496 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:15.967156 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:15.967128 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:33:16.040372 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.040339 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 14:33:16.040930 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.040481 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 14:33:16.040930 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.040527 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 14:33:16.042651 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:16.042632 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:16.143018 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:16.142943 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:16.144538 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.144511 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 14:28:16 +0000 UTC" deadline="2028-04-22 10:24:18.433609612 +0000 UTC" Jul 07 14:33:16.144603 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.144540 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15715h51m2.289074768s" Jul 07 14:33:16.243198 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.243174 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:33:16.243315 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:16.243174 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:16.246858 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.246842 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:33:16.246951 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.246883 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:33:16.246951 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.246900 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:33:16.343461 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:16.343432 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-146.ec2.internal\" not found" Jul 07 14:33:16.344401 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.344383 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:33:16.397850 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.397791 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" Jul 07 14:33:16.398430 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:16.398394 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9df984a94f344c9e5dde59a74f6e184.slice/crio-0f0fb2a73ad86ca6919ef63648b9e9d12afacd174f15a29946dd6be9f8ec5e74 WatchSource:0}: Error finding container 0f0fb2a73ad86ca6919ef63648b9e9d12afacd174f15a29946dd6be9f8ec5e74: Status 404 returned error can't find the container with id 0f0fb2a73ad86ca6919ef63648b9e9d12afacd174f15a29946dd6be9f8ec5e74 Jul 07 14:33:16.402757 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.402741 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:33:16.412498 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.412478 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 14:33:16.413509 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.413496 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-146.ec2.internal" Jul 07 14:33:16.423234 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.423209 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 14:33:16.444369 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:16.444351 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:33:16.703478 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:16.703448 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c897cdd111a3fcc4bf6f54f3d25813.slice/crio-3cae97b3af66e82df5158c887e0775839191f8a117b40c69bf2e0d1875ffc12c WatchSource:0}: Error finding container 3cae97b3af66e82df5158c887e0775839191f8a117b40c69bf2e0d1875ffc12c: Status 404 returned error can't find the container with id 3cae97b3af66e82df5158c887e0775839191f8a117b40c69bf2e0d1875ffc12c Jul 07 14:33:17.076657 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.076581 2581 apiserver.go:52] "Watching apiserver" Jul 07 14:33:17.089075 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.089056 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 14:33:17.090381 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.090358 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk","openshift-cluster-node-tuning-operator/tuned-l5qx6","openshift-dns/node-resolver-92t44","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal","openshift-multus/network-metrics-daemon-mzxzk","openshift-network-operator/iptables-alerter-qlwmh","kube-system/global-pull-secret-syncer-xb5ll","openshift-image-registry/node-ca-6hfkd","openshift-multus/multus-additional-cni-plugins-xnhfr","openshift-multus/multus-x2bcs","openshift-network-diagnostics/network-check-target-jrxsr","openshift-ovn-kubernetes/ovnkube-node-b4k7w","kube-system/konnectivity-agent-fcqkx","kube-system/kube-apiserver-proxy-ip-10-0-143-146.ec2.internal"] Jul 07 14:33:17.092471 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.092452 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.093686 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.093666 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:17.094810 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.094786 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-92t44" Jul 07 14:33:17.095539 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.095518 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 14:33:17.095539 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.095535 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 14:33:17.095694 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.095550 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 14:33:17.095694 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.095571 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bgh2z\"" Jul 07 14:33:17.095694 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.095537 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 14:33:17.095985 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.095972 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6hfkd" Jul 07 14:33:17.096427 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.096408 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 14:33:17.096569 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.096476 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 14:33:17.096569 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.096528 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lrzmx\"" Jul 07 14:33:17.097046 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.097027 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 14:33:17.097226 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.097177 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:17.097293 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.097242 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:17.097361 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.097299 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 14:33:17.097458 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.097443 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7t9zx\"" Jul 07 14:33:17.097685 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.097669 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 14:33:17.098415 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.098393 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qlwmh" Jul 07 14:33:17.098518 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.098466 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 14:33:17.098626 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.098599 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 14:33:17.099030 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.099004 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 14:33:17.099128 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.099082 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-clj79\"" Jul 07 14:33:17.099847 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.099811 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.100819 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.100800 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:33:17.100924 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.100909 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jmsfj\"" Jul 07 14:33:17.101164 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.101143 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 14:33:17.101271 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.101254 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:17.101345 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.101284 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 14:33:17.101345 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.101315 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:17.102539 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.102521 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 14:33:17.102539 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.102533 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 14:33:17.102680 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.102615 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.102833 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.102817 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kj9xr\"" Jul 07 14:33:17.104034 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.104015 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.105225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.105204 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:33:17.105334 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.105266 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 14:33:17.105809 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.105786 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:17.105926 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.105854 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kpd5w\"" Jul 07 14:33:17.105926 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.105870 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:17.106763 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.106277 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 14:33:17.107128 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.107102 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xkhb6\"" Jul 07 14:33:17.107282 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.107262 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 14:33:17.107361 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.107344 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 14:33:17.107560 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.107512 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ecd6676-11b6-49ad-b15f-e08fbc43c0b6-tmp-dir\") pod \"node-resolver-92t44\" (UID: \"9ecd6676-11b6-49ad-b15f-e08fbc43c0b6\") " pod="openshift-dns/node-resolver-92t44" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108047 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2904380c-b1fe-4350-83ce-2c0ec2859ed7-iptables-alerter-script\") pod \"iptables-alerter-qlwmh\" (UID: \"2904380c-b1fe-4350-83ce-2c0ec2859ed7\") " pod="openshift-network-operator/iptables-alerter-qlwmh" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108087 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnvjp\" (UniqueName: \"kubernetes.io/projected/2904380c-b1fe-4350-83ce-2c0ec2859ed7-kube-api-access-fnvjp\") pod \"iptables-alerter-qlwmh\" (UID: \"2904380c-b1fe-4350-83ce-2c0ec2859ed7\") " pod="openshift-network-operator/iptables-alerter-qlwmh" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108123 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-socket-dir-parent\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108155 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f77q\" (UniqueName: \"kubernetes.io/projected/17ef7399-7fb7-4446-b7b5-c85cc6d7e43e-kube-api-access-6f77q\") pod \"node-ca-6hfkd\" (UID: \"17ef7399-7fb7-4446-b7b5-c85cc6d7e43e\") " pod="openshift-image-registry/node-ca-6hfkd" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108191 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108225 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z8kv\" (UniqueName: \"kubernetes.io/projected/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-kube-api-access-9z8kv\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108284 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-cni-dir\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108323 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-run-k8s-cni-cncf-io\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108371 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-var-lib-cni-bin\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108407 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108451 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a097e24-d671-4a5a-96c4-c3f50e38d5e3-konnectivity-ca\") pod \"konnectivity-agent-fcqkx\" (UID: \"4a097e24-d671-4a5a-96c4-c3f50e38d5e3\") " pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108499 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-os-release\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108530 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-run-netns\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108570 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-conf-dir\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108606 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17ef7399-7fb7-4446-b7b5-c85cc6d7e43e-serviceca\") pod \"node-ca-6hfkd\" (UID: \"17ef7399-7fb7-4446-b7b5-c85cc6d7e43e\") " pod="openshift-image-registry/node-ca-6hfkd" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108687 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2904380c-b1fe-4350-83ce-2c0ec2859ed7-host-slash\") pod \"iptables-alerter-qlwmh\" (UID: \"2904380c-b1fe-4350-83ce-2c0ec2859ed7\") " pod="openshift-network-operator/iptables-alerter-qlwmh" Jul 07 14:33:17.108946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108724 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-var-lib-kubelet\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108771 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-hostroot\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108806 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a097e24-d671-4a5a-96c4-c3f50e38d5e3-agent-certs\") pod \"konnectivity-agent-fcqkx\" (UID: \"4a097e24-d671-4a5a-96c4-c3f50e38d5e3\") " pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108838 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ecd6676-11b6-49ad-b15f-e08fbc43c0b6-hosts-file\") pod \"node-resolver-92t44\" (UID: \"9ecd6676-11b6-49ad-b15f-e08fbc43c0b6\") " pod="openshift-dns/node-resolver-92t44" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.108865 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfb9g\" (UniqueName: \"kubernetes.io/projected/9ecd6676-11b6-49ad-b15f-e08fbc43c0b6-kube-api-access-bfb9g\") pod \"node-resolver-92t44\" (UID: \"9ecd6676-11b6-49ad-b15f-e08fbc43c0b6\") " pod="openshift-dns/node-resolver-92t44" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.109035 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-system-cni-dir\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.109090 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-cnibin\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.109135 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-cni-binary-copy\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.109179 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-var-lib-cni-multus\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.109230 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-etc-kubernetes\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.109286 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmgzl\" (UniqueName: \"kubernetes.io/projected/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-kube-api-access-mmgzl\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.109317 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ef7399-7fb7-4446-b7b5-c85cc6d7e43e-host\") pod \"node-ca-6hfkd\" (UID: \"17ef7399-7fb7-4446-b7b5-c85cc6d7e43e\") " pod="openshift-image-registry/node-ca-6hfkd" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.109371 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-daemon-config\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.109788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.109406 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-run-multus-certs\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.110837 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.110815 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 14:33:17.110953 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.110933 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 14:33:17.111112 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.111085 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tw54v\"" Jul 07 14:33:17.112427 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.112410 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 14:33:17.112598 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.112582 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 14:33:17.112598 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.112587 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 14:33:17.112764 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.112677 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 14:33:17.113942 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.113921 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 14:33:17.145484 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.145463 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mq2kh" Jul 07 14:33:17.155491 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.155463 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mq2kh" Jul 07 14:33:17.198731 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.198705 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 14:33:17.209663 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209633 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.209772 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209677 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f77q\" (UniqueName: \"kubernetes.io/projected/17ef7399-7fb7-4446-b7b5-c85cc6d7e43e-kube-api-access-6f77q\") pod \"node-ca-6hfkd\" (UID: \"17ef7399-7fb7-4446-b7b5-c85cc6d7e43e\") " pod="openshift-image-registry/node-ca-6hfkd" Jul 07 14:33:17.209772 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209704 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z8kv\" (UniqueName: \"kubernetes.io/projected/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-kube-api-access-9z8kv\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:17.209772 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209730 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-modprobe-d\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.209772 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209756 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.210011 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209781 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-kubelet\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.210011 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209807 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-cni-bin\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.210011 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209832 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-system-cni-dir\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.210011 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209886 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-cni-dir\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.210011 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209912 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-var-lib-cni-bin\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.210011 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209939 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj9kl\" (UniqueName: \"kubernetes.io/projected/ea46ddfe-55c0-47f0-8a9a-911f295bf044-kube-api-access-fj9kl\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.210011 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209967 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-os-release\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.210011 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.209994 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-sysconfig\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210019 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-sysctl-conf\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210025 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-var-lib-cni-bin\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210044 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-sys\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210052 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-os-release\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210082 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-cni-dir\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210069 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea46ddfe-55c0-47f0-8a9a-911f295bf044-ovn-node-metrics-cert\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210131 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-var-lib-kubelet\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210154 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9fc9\" (UniqueName: \"kubernetes.io/projected/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-kube-api-access-w9fc9\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210173 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-node-log\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210195 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ecd6676-11b6-49ad-b15f-e08fbc43c0b6-hosts-file\") pod \"node-resolver-92t44\" (UID: \"9ecd6676-11b6-49ad-b15f-e08fbc43c0b6\") " pod="openshift-dns/node-resolver-92t44" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210276 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ecd6676-11b6-49ad-b15f-e08fbc43c0b6-hosts-file\") pod \"node-resolver-92t44\" (UID: \"9ecd6676-11b6-49ad-b15f-e08fbc43c0b6\") " pod="openshift-dns/node-resolver-92t44" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210311 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-system-cni-dir\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.210337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210338 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/91b21080-c287-47e7-9829-601c1be484f2-dbus\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210357 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-registration-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210374 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-run-netns\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210400 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-run-systemd\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210402 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-system-cni-dir\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210417 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-cnibin\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-cni-binary-copy\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210449 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-run\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210465 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-host\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210480 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-etc-openvswitch\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210497 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-run-ovn\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210522 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ef7399-7fb7-4446-b7b5-c85cc6d7e43e-host\") pod \"node-ca-6hfkd\" (UID: \"17ef7399-7fb7-4446-b7b5-c85cc6d7e43e\") " pod="openshift-image-registry/node-ca-6hfkd" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210545 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbgvd\" (UniqueName: \"kubernetes.io/projected/09b3e817-b4bd-4864-80f1-835e11bd58f3-kube-api-access-xbgvd\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210570 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ecd6676-11b6-49ad-b15f-e08fbc43c0b6-tmp-dir\") pod \"node-resolver-92t44\" (UID: \"9ecd6676-11b6-49ad-b15f-e08fbc43c0b6\") " pod="openshift-dns/node-resolver-92t44" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210591 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2904380c-b1fe-4350-83ce-2c0ec2859ed7-iptables-alerter-script\") pod \"iptables-alerter-qlwmh\" (UID: \"2904380c-b1fe-4350-83ce-2c0ec2859ed7\") " pod="openshift-network-operator/iptables-alerter-qlwmh" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210619 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-socket-dir-parent\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210655 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ef7399-7fb7-4446-b7b5-c85cc6d7e43e-host\") pod \"node-ca-6hfkd\" (UID: \"17ef7399-7fb7-4446-b7b5-c85cc6d7e43e\") " pod="openshift-image-registry/node-ca-6hfkd" Jul 07 14:33:17.210943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210664 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/91b21080-c287-47e7-9829-601c1be484f2-kubelet-config\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210687 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-sysctl-d\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210749 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210730 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-socket-dir-parent\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210778 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-run-openvswitch\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea46ddfe-55c0-47f0-8a9a-911f295bf044-ovnkube-script-lib\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210871 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210920 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-lib-modules\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210930 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ecd6676-11b6-49ad-b15f-e08fbc43c0b6-tmp-dir\") pod \"node-resolver-92t44\" (UID: \"9ecd6676-11b6-49ad-b15f-e08fbc43c0b6\") " pod="openshift-dns/node-resolver-92t44" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210953 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-cni-binary-copy\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.210993 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzhxz\" (UniqueName: \"kubernetes.io/projected/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-kube-api-access-bzhxz\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.211005 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211031 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-cni-binary-copy\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211032 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-run-k8s-cni-cncf-io\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211068 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-sys-fs\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211074 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-run-k8s-cni-cncf-io\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.211565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211102 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-var-lib-openvswitch\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.211148 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs podName:0f5b7926-67d9-4e32-bc4e-0c14a42118ea nodeName:}" failed. No retries permitted until 2026-07-07 14:33:17.711132015 +0000 UTC m=+2.986215421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs") pod "network-metrics-daemon-mzxzk" (UID: "0f5b7926-67d9-4e32-bc4e-0c14a42118ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211168 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2904380c-b1fe-4350-83ce-2c0ec2859ed7-iptables-alerter-script\") pod \"iptables-alerter-qlwmh\" (UID: \"2904380c-b1fe-4350-83ce-2c0ec2859ed7\") " pod="openshift-network-operator/iptables-alerter-qlwmh" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211175 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-os-release\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211201 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a097e24-d671-4a5a-96c4-c3f50e38d5e3-konnectivity-ca\") pod \"konnectivity-agent-fcqkx\" (UID: \"4a097e24-d671-4a5a-96c4-c3f50e38d5e3\") " pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211246 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-run-netns\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211353 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-conf-dir\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211365 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-run-netns\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211401 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-tuned\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211407 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-conf-dir\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211437 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211462 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211482 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211520 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17ef7399-7fb7-4446-b7b5-c85cc6d7e43e-serviceca\") pod \"node-ca-6hfkd\" (UID: \"17ef7399-7fb7-4446-b7b5-c85cc6d7e43e\") " pod="openshift-image-registry/node-ca-6hfkd" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211551 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2904380c-b1fe-4350-83ce-2c0ec2859ed7-host-slash\") pod \"iptables-alerter-qlwmh\" (UID: \"2904380c-b1fe-4350-83ce-2c0ec2859ed7\") " pod="openshift-network-operator/iptables-alerter-qlwmh" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211665 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2904380c-b1fe-4350-83ce-2c0ec2859ed7-host-slash\") pod \"iptables-alerter-qlwmh\" (UID: \"2904380c-b1fe-4350-83ce-2c0ec2859ed7\") " pod="openshift-network-operator/iptables-alerter-qlwmh" Jul 07 14:33:17.212225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211666 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-var-lib-kubelet\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211699 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-hostroot\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211705 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-var-lib-kubelet\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211714 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-tmp\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211712 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a097e24-d671-4a5a-96c4-c3f50e38d5e3-konnectivity-ca\") pod \"konnectivity-agent-fcqkx\" (UID: \"4a097e24-d671-4a5a-96c4-c3f50e38d5e3\") " pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211748 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-hostroot\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211746 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a097e24-d671-4a5a-96c4-c3f50e38d5e3-agent-certs\") pod \"konnectivity-agent-fcqkx\" (UID: \"4a097e24-d671-4a5a-96c4-c3f50e38d5e3\") " pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211789 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfb9g\" (UniqueName: \"kubernetes.io/projected/9ecd6676-11b6-49ad-b15f-e08fbc43c0b6-kube-api-access-bfb9g\") pod \"node-resolver-92t44\" (UID: \"9ecd6676-11b6-49ad-b15f-e08fbc43c0b6\") " pod="openshift-dns/node-resolver-92t44" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211815 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-systemd\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211852 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-socket-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211873 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-cni-netd\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211894 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-cnibin\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211915 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-var-lib-cni-multus\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211917 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17ef7399-7fb7-4446-b7b5-c85cc6d7e43e-serviceca\") pod \"node-ca-6hfkd\" (UID: \"17ef7399-7fb7-4446-b7b5-c85cc6d7e43e\") " pod="openshift-image-registry/node-ca-6hfkd" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211939 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-etc-kubernetes\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211968 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-var-lib-cni-multus\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211988 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-cnibin\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.211990 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-etc-kubernetes\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.212792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmgzl\" (UniqueName: \"kubernetes.io/projected/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-kube-api-access-mmgzl\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212064 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8pn6\" (UniqueName: \"kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6\") pod \"network-check-target-jrxsr\" (UID: \"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4\") " pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212084 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea46ddfe-55c0-47f0-8a9a-911f295bf044-ovnkube-config\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212102 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-daemon-config\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212118 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-run-multus-certs\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212134 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212160 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-kubernetes\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212141 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212172 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-host-run-multus-certs\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212177 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-slash\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvjp\" (UniqueName: \"kubernetes.io/projected/2904380c-b1fe-4350-83ce-2c0ec2859ed7-kube-api-access-fnvjp\") pod \"iptables-alerter-qlwmh\" (UID: \"2904380c-b1fe-4350-83ce-2c0ec2859ed7\") " pod="openshift-network-operator/iptables-alerter-qlwmh" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212218 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-device-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212238 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-systemd-units\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212253 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-log-socket\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212268 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea46ddfe-55c0-47f0-8a9a-911f295bf044-env-overrides\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.213351 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.212519 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-multus-daemon-config\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.215545 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.215524 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a097e24-d671-4a5a-96c4-c3f50e38d5e3-agent-certs\") pod \"konnectivity-agent-fcqkx\" (UID: \"4a097e24-d671-4a5a-96c4-c3f50e38d5e3\") " pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:17.222737 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.222716 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfb9g\" (UniqueName: \"kubernetes.io/projected/9ecd6676-11b6-49ad-b15f-e08fbc43c0b6-kube-api-access-bfb9g\") pod \"node-resolver-92t44\" (UID: \"9ecd6676-11b6-49ad-b15f-e08fbc43c0b6\") " pod="openshift-dns/node-resolver-92t44" Jul 07 14:33:17.223181 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.223152 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f77q\" (UniqueName: \"kubernetes.io/projected/17ef7399-7fb7-4446-b7b5-c85cc6d7e43e-kube-api-access-6f77q\") pod \"node-ca-6hfkd\" (UID: \"17ef7399-7fb7-4446-b7b5-c85cc6d7e43e\") " pod="openshift-image-registry/node-ca-6hfkd" Jul 07 14:33:17.223377 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.223359 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z8kv\" (UniqueName: \"kubernetes.io/projected/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-kube-api-access-9z8kv\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:17.225529 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.225508 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmgzl\" (UniqueName: \"kubernetes.io/projected/b1cea48d-ba43-4419-9966-bc2b80b2fcc0-kube-api-access-mmgzl\") pod \"multus-x2bcs\" (UID: \"b1cea48d-ba43-4419-9966-bc2b80b2fcc0\") " pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.226588 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.226573 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnvjp\" (UniqueName: \"kubernetes.io/projected/2904380c-b1fe-4350-83ce-2c0ec2859ed7-kube-api-access-fnvjp\") pod \"iptables-alerter-qlwmh\" (UID: \"2904380c-b1fe-4350-83ce-2c0ec2859ed7\") " pod="openshift-network-operator/iptables-alerter-qlwmh" Jul 07 14:33:17.241131 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.241092 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-146.ec2.internal" event={"ID":"00c897cdd111a3fcc4bf6f54f3d25813","Type":"ContainerStarted","Data":"3cae97b3af66e82df5158c887e0775839191f8a117b40c69bf2e0d1875ffc12c"} Jul 07 14:33:17.242095 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.242075 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" event={"ID":"d9df984a94f344c9e5dde59a74f6e184","Type":"ContainerStarted","Data":"0f0fb2a73ad86ca6919ef63648b9e9d12afacd174f15a29946dd6be9f8ec5e74"} Jul 07 14:33:17.313084 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313049 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-tuned\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.313084 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313090 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.313316 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313112 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.313316 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313131 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.313316 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313155 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-tmp\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.313316 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313202 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.313316 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313283 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.313601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313346 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-systemd\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.313601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313421 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-socket-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.313601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313450 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-cni-netd\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.313601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313483 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pn6\" (UniqueName: \"kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6\") pod \"network-check-target-jrxsr\" (UID: \"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4\") " pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:17.313601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313493 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-systemd\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.313601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313510 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea46ddfe-55c0-47f0-8a9a-911f295bf044-ovnkube-config\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.313601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313562 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:17.313601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313591 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-kubernetes\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313619 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-slash\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313647 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-device-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313670 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-systemd-units\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.313672 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313694 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-log-socket\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313719 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea46ddfe-55c0-47f0-8a9a-911f295bf044-env-overrides\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.313766 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret podName:91b21080-c287-47e7-9829-601c1be484f2 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:17.813741154 +0000 UTC m=+3.088824564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret") pod "global-pull-secret-syncer-xb5ll" (UID: "91b21080-c287-47e7-9829-601c1be484f2") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313787 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313801 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313818 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-modprobe-d\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313859 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313909 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-kubelet\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313957 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-modprobe-d\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313956 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-cni-netd\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313980 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-cni-bin\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314002 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.314032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314014 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-system-cni-dir\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314027 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-kubernetes\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313958 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-kubelet\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.313591 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-socket-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314059 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-systemd-units\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314072 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-cni-bin\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314094 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj9kl\" (UniqueName: \"kubernetes.io/projected/ea46ddfe-55c0-47f0-8a9a-911f295bf044-kube-api-access-fj9kl\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314116 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-slash\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314116 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea46ddfe-55c0-47f0-8a9a-911f295bf044-ovnkube-config\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314175 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-device-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314218 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-system-cni-dir\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314227 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-sysconfig\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314266 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-sysctl-conf\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314291 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-sys\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314268 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-log-socket\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314359 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea46ddfe-55c0-47f0-8a9a-911f295bf044-ovn-node-metrics-cert\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314391 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-var-lib-kubelet\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.314670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314412 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9fc9\" (UniqueName: \"kubernetes.io/projected/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-kube-api-access-w9fc9\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314423 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-sysconfig\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-node-log\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314448 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-sysctl-conf\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314468 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314485 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-var-lib-kubelet\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314490 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-sys\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314496 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/91b21080-c287-47e7-9829-601c1be484f2-dbus\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314538 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-node-log\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314562 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/91b21080-c287-47e7-9829-601c1be484f2-dbus\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314566 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-registration-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314593 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-run-netns\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314618 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-run-systemd\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-cnibin\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314650 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314661 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea46ddfe-55c0-47f0-8a9a-911f295bf044-env-overrides\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314670 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-run\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314696 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-host-run-netns\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.315386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314707 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-run\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314742 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-host\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314766 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-run-systemd\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314795 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-registration-dir\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314832 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-etc-openvswitch\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314848 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-host\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314859 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-run-ovn\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314889 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-cnibin\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314951 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbgvd\" (UniqueName: \"kubernetes.io/projected/09b3e817-b4bd-4864-80f1-835e11bd58f3-kube-api-access-xbgvd\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.314974 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-etc-openvswitch\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315021 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/91b21080-c287-47e7-9829-601c1be484f2-kubelet-config\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315066 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/91b21080-c287-47e7-9829-601c1be484f2-kubelet-config\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315067 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-sysctl-d\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315104 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315190 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-sysctl-d\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315218 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-run-ovn\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315235 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-run-openvswitch\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.317041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea46ddfe-55c0-47f0-8a9a-911f295bf044-ovnkube-script-lib\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315275 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315342 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-run-openvswitch\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315391 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-lib-modules\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315417 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-cni-binary-copy\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315445 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzhxz\" (UniqueName: \"kubernetes.io/projected/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-kube-api-access-bzhxz\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315443 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-sys-fs\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315505 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-var-lib-openvswitch\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315530 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-os-release\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315549 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-lib-modules\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315621 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-os-release\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315620 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09b3e817-b4bd-4864-80f1-835e11bd58f3-sys-fs\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315701 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea46ddfe-55c0-47f0-8a9a-911f295bf044-var-lib-openvswitch\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315835 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea46ddfe-55c0-47f0-8a9a-911f295bf044-ovnkube-script-lib\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315930 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-tmp\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.315974 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-cni-binary-copy\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.316146 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-etc-tuned\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.317796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.317573 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:33:17.318428 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.317676 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea46ddfe-55c0-47f0-8a9a-911f295bf044-ovn-node-metrics-cert\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.323852 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.323815 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:33:17.323958 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.323857 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:33:17.323958 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.323873 2581 projected.go:196] Error preparing data for projected volume kube-api-access-z8pn6 for pod openshift-network-diagnostics/network-check-target-jrxsr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:17.323958 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.323952 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6 podName:4e7b899b-9bd2-4fb6-907c-69e88b59c1b4 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:17.823936093 +0000 UTC m=+3.099019515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z8pn6" (UniqueName: "kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6") pod "network-check-target-jrxsr" (UID: "4e7b899b-9bd2-4fb6-907c-69e88b59c1b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:17.326509 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.326482 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj9kl\" (UniqueName: \"kubernetes.io/projected/ea46ddfe-55c0-47f0-8a9a-911f295bf044-kube-api-access-fj9kl\") pod \"ovnkube-node-b4k7w\" (UID: \"ea46ddfe-55c0-47f0-8a9a-911f295bf044\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.328284 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.328229 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzhxz\" (UniqueName: \"kubernetes.io/projected/03c12bd0-a596-4c34-977c-3dea4a7d4fe0-kube-api-access-bzhxz\") pod \"multus-additional-cni-plugins-xnhfr\" (UID: \"03c12bd0-a596-4c34-977c-3dea4a7d4fe0\") " pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.328643 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.328622 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9fc9\" (UniqueName: \"kubernetes.io/projected/bb461e6a-9c87-4833-8a9e-ef0e6e9c18da-kube-api-access-w9fc9\") pod \"tuned-l5qx6\" (UID: \"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da\") " pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.328731 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.328701 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbgvd\" (UniqueName: \"kubernetes.io/projected/09b3e817-b4bd-4864-80f1-835e11bd58f3-kube-api-access-xbgvd\") pod \"aws-ebs-csi-driver-node-8sqmk\" (UID: \"09b3e817-b4bd-4864-80f1-835e11bd58f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.403287 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.403253 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x2bcs" Jul 07 14:33:17.412154 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.412128 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:17.412659 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:17.412630 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1cea48d_ba43_4419_9966_bc2b80b2fcc0.slice/crio-ee42a3cc20734bfc80d9645c7b31153a90a513f6fad87838b782058f076d47bc WatchSource:0}: Error finding container ee42a3cc20734bfc80d9645c7b31153a90a513f6fad87838b782058f076d47bc: Status 404 returned error can't find the container with id ee42a3cc20734bfc80d9645c7b31153a90a513f6fad87838b782058f076d47bc Jul 07 14:33:17.419788 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.419766 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-92t44" Jul 07 14:33:17.421173 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:17.421150 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a097e24_d671_4a5a_96c4_c3f50e38d5e3.slice/crio-7fdd6b1a5c536c70a58da6ab35805c4de0889f25df5ebf068277eb59fa6e076b WatchSource:0}: Error finding container 7fdd6b1a5c536c70a58da6ab35805c4de0889f25df5ebf068277eb59fa6e076b: Status 404 returned error can't find the container with id 7fdd6b1a5c536c70a58da6ab35805c4de0889f25df5ebf068277eb59fa6e076b Jul 07 14:33:17.423539 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.423457 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6hfkd" Jul 07 14:33:17.430996 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.430950 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qlwmh" Jul 07 14:33:17.433179 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:17.433086 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ecd6676_11b6_49ad_b15f_e08fbc43c0b6.slice/crio-6266b8a83332c7bf4d35b943f85c444425b1d4e6e348a1e79081f3802c4cf381 WatchSource:0}: Error finding container 6266b8a83332c7bf4d35b943f85c444425b1d4e6e348a1e79081f3802c4cf381: Status 404 returned error can't find the container with id 6266b8a83332c7bf4d35b943f85c444425b1d4e6e348a1e79081f3802c4cf381 Jul 07 14:33:17.433457 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:17.433431 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ef7399_7fb7_4446_b7b5_c85cc6d7e43e.slice/crio-f46527c7b72860ec6a447873964b3accbcd72e1254ad0bc2f1855b8e07f873f4 WatchSource:0}: Error finding container f46527c7b72860ec6a447873964b3accbcd72e1254ad0bc2f1855b8e07f873f4: Status 404 returned error can't find the container with id f46527c7b72860ec6a447873964b3accbcd72e1254ad0bc2f1855b8e07f873f4 Jul 07 14:33:17.437207 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.437163 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" Jul 07 14:33:17.442553 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:17.442279 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2904380c_b1fe_4350_83ce_2c0ec2859ed7.slice/crio-26e45b977e8bfaf64cc690e52d53ee078fb1bde00a8fbd9c2f87e0a9435e37ae WatchSource:0}: Error finding container 26e45b977e8bfaf64cc690e52d53ee078fb1bde00a8fbd9c2f87e0a9435e37ae: Status 404 returned error can't find the container with id 26e45b977e8bfaf64cc690e52d53ee078fb1bde00a8fbd9c2f87e0a9435e37ae Jul 07 14:33:17.443355 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.443333 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" Jul 07 14:33:17.449633 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.449609 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" Jul 07 14:33:17.449915 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:17.449881 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c12bd0_a596_4c34_977c_3dea4a7d4fe0.slice/crio-afc2d50861cc18a430030225e428d56eef07fae9f920b32e4db9633c721deff9 WatchSource:0}: Error finding container afc2d50861cc18a430030225e428d56eef07fae9f920b32e4db9633c721deff9: Status 404 returned error can't find the container with id afc2d50861cc18a430030225e428d56eef07fae9f920b32e4db9633c721deff9 Jul 07 14:33:17.453223 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.453206 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:17.454452 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:17.454430 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb461e6a_9c87_4833_8a9e_ef0e6e9c18da.slice/crio-e77fb32c2a54aab96c31ef9db70d48ff6444f71a5e23afab255c2ddd136a8232 WatchSource:0}: Error finding container e77fb32c2a54aab96c31ef9db70d48ff6444f71a5e23afab255c2ddd136a8232: Status 404 returned error can't find the container with id e77fb32c2a54aab96c31ef9db70d48ff6444f71a5e23afab255c2ddd136a8232 Jul 07 14:33:17.463323 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:17.463290 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b3e817_b4bd_4864_80f1_835e11bd58f3.slice/crio-32abbfa1057619139d229e7393a6f22b2848206c130158ffaad98e0d34e88eb7 WatchSource:0}: Error finding container 32abbfa1057619139d229e7393a6f22b2848206c130158ffaad98e0d34e88eb7: Status 404 returned error can't find the container with id 32abbfa1057619139d229e7393a6f22b2848206c130158ffaad98e0d34e88eb7 Jul 07 14:33:17.466467 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:17.466442 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea46ddfe_55c0_47f0_8a9a_911f295bf044.slice/crio-129df84b50a986be7866811e6cb324aa7af8fb9ff40ef0cbad93cf86d6b8149d WatchSource:0}: Error finding container 129df84b50a986be7866811e6cb324aa7af8fb9ff40ef0cbad93cf86d6b8149d: Status 404 returned error can't find the container with id 129df84b50a986be7866811e6cb324aa7af8fb9ff40ef0cbad93cf86d6b8149d Jul 07 14:33:17.720151 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.720113 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:17.720370 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.720304 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:17.720434 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.720408 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs podName:0f5b7926-67d9-4e32-bc4e-0c14a42118ea nodeName:}" failed. No retries permitted until 2026-07-07 14:33:18.720370929 +0000 UTC m=+3.995454336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs") pod "network-metrics-daemon-mzxzk" (UID: "0f5b7926-67d9-4e32-bc4e-0c14a42118ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:17.821488 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.821450 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:17.821658 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.821562 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:17.821658 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.821621 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret podName:91b21080-c287-47e7-9829-601c1be484f2 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:18.821604588 +0000 UTC m=+4.096687995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret") pod "global-pull-secret-syncer-xb5ll" (UID: "91b21080-c287-47e7-9829-601c1be484f2") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:17.922732 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:17.922680 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pn6\" (UniqueName: \"kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6\") pod \"network-check-target-jrxsr\" (UID: \"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4\") " pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:17.922906 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.922876 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:33:17.922906 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.922900 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:33:17.923021 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.922911 2581 projected.go:196] Error preparing data for projected volume kube-api-access-z8pn6 for pod openshift-network-diagnostics/network-check-target-jrxsr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:17.923021 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:17.922973 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6 podName:4e7b899b-9bd2-4fb6-907c-69e88b59c1b4 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:18.922958485 +0000 UTC m=+4.198041908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8pn6" (UniqueName: "kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6") pod "network-check-target-jrxsr" (UID: "4e7b899b-9bd2-4fb6-907c-69e88b59c1b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:18.157787 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.157739 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:28:17 +0000 UTC" deadline="2028-03-05 00:03:28.243962353 +0000 UTC" Jul 07 14:33:18.157787 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.157781 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14553h30m10.086186283s" Jul 07 14:33:18.249545 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.249414 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6hfkd" event={"ID":"17ef7399-7fb7-4446-b7b5-c85cc6d7e43e","Type":"ContainerStarted","Data":"f46527c7b72860ec6a447873964b3accbcd72e1254ad0bc2f1855b8e07f873f4"} Jul 07 14:33:18.251258 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.251178 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-92t44" event={"ID":"9ecd6676-11b6-49ad-b15f-e08fbc43c0b6","Type":"ContainerStarted","Data":"6266b8a83332c7bf4d35b943f85c444425b1d4e6e348a1e79081f3802c4cf381"} Jul 07 14:33:18.252617 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.252542 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" event={"ID":"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da","Type":"ContainerStarted","Data":"e77fb32c2a54aab96c31ef9db70d48ff6444f71a5e23afab255c2ddd136a8232"} Jul 07 14:33:18.254528 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.254461 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fcqkx" event={"ID":"4a097e24-d671-4a5a-96c4-c3f50e38d5e3","Type":"ContainerStarted","Data":"7fdd6b1a5c536c70a58da6ab35805c4de0889f25df5ebf068277eb59fa6e076b"} Jul 07 14:33:18.258322 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.258253 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x2bcs" event={"ID":"b1cea48d-ba43-4419-9966-bc2b80b2fcc0","Type":"ContainerStarted","Data":"ee42a3cc20734bfc80d9645c7b31153a90a513f6fad87838b782058f076d47bc"} Jul 07 14:33:18.259914 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.259755 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" event={"ID":"ea46ddfe-55c0-47f0-8a9a-911f295bf044","Type":"ContainerStarted","Data":"129df84b50a986be7866811e6cb324aa7af8fb9ff40ef0cbad93cf86d6b8149d"} Jul 07 14:33:18.263934 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.263831 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" event={"ID":"09b3e817-b4bd-4864-80f1-835e11bd58f3","Type":"ContainerStarted","Data":"32abbfa1057619139d229e7393a6f22b2848206c130158ffaad98e0d34e88eb7"} Jul 07 14:33:18.266936 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.266845 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" event={"ID":"03c12bd0-a596-4c34-977c-3dea4a7d4fe0","Type":"ContainerStarted","Data":"afc2d50861cc18a430030225e428d56eef07fae9f920b32e4db9633c721deff9"} Jul 07 14:33:18.269245 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.269179 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qlwmh" event={"ID":"2904380c-b1fe-4350-83ce-2c0ec2859ed7","Type":"ContainerStarted","Data":"26e45b977e8bfaf64cc690e52d53ee078fb1bde00a8fbd9c2f87e0a9435e37ae"} Jul 07 14:33:18.729617 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.729583 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:18.729809 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:18.729728 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:18.729809 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:18.729790 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs podName:0f5b7926-67d9-4e32-bc4e-0c14a42118ea nodeName:}" failed. No retries permitted until 2026-07-07 14:33:20.729771275 +0000 UTC m=+6.004854680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs") pod "network-metrics-daemon-mzxzk" (UID: "0f5b7926-67d9-4e32-bc4e-0c14a42118ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:18.830323 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.830285 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:18.830521 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:18.830446 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:18.830585 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:18.830549 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret podName:91b21080-c287-47e7-9829-601c1be484f2 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:20.830529771 +0000 UTC m=+6.105613184 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret") pod "global-pull-secret-syncer-xb5ll" (UID: "91b21080-c287-47e7-9829-601c1be484f2") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:18.933289 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.933254 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pn6\" (UniqueName: \"kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6\") pod \"network-check-target-jrxsr\" (UID: \"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4\") " pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:18.933491 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:18.933457 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:33:18.933491 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:18.933485 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:33:18.933624 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:18.933500 2581 projected.go:196] Error preparing data for projected volume kube-api-access-z8pn6 for pod openshift-network-diagnostics/network-check-target-jrxsr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:18.933624 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:18.933564 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6 podName:4e7b899b-9bd2-4fb6-907c-69e88b59c1b4 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:20.933542698 +0000 UTC m=+6.208626123 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8pn6" (UniqueName: "kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6") pod "network-check-target-jrxsr" (UID: "4e7b899b-9bd2-4fb6-907c-69e88b59c1b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:18.987810 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:18.987723 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:33:19.158079 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:19.158006 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:28:17 +0000 UTC" deadline="2028-01-19 07:37:45.61785978 +0000 UTC" Jul 07 14:33:19.158079 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:19.158077 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13457h4m26.459787415s" Jul 07 14:33:19.236330 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:19.236298 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:19.236517 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:19.236449 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:19.236926 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:19.236889 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:19.237034 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:19.237012 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:19.237318 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:19.237183 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:19.237318 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:19.237270 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:20.751364 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:20.751318 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:20.751981 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:20.751483 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:20.751981 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:20.751554 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs podName:0f5b7926-67d9-4e32-bc4e-0c14a42118ea nodeName:}" failed. No retries permitted until 2026-07-07 14:33:24.751532922 +0000 UTC m=+10.026616335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs") pod "network-metrics-daemon-mzxzk" (UID: "0f5b7926-67d9-4e32-bc4e-0c14a42118ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:20.852364 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:20.852326 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:20.852554 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:20.852491 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:20.852620 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:20.852560 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret podName:91b21080-c287-47e7-9829-601c1be484f2 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:24.852539398 +0000 UTC m=+10.127622808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret") pod "global-pull-secret-syncer-xb5ll" (UID: "91b21080-c287-47e7-9829-601c1be484f2") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:20.953232 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:20.953191 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pn6\" (UniqueName: \"kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6\") pod \"network-check-target-jrxsr\" (UID: \"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4\") " pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:20.953429 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:20.953407 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:33:20.953429 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:20.953428 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:33:20.953531 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:20.953442 2581 projected.go:196] Error preparing data for projected volume kube-api-access-z8pn6 for pod openshift-network-diagnostics/network-check-target-jrxsr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:20.953531 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:20.953502 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6 podName:4e7b899b-9bd2-4fb6-907c-69e88b59c1b4 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:24.953482245 +0000 UTC m=+10.228565667 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8pn6" (UniqueName: "kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6") pod "network-check-target-jrxsr" (UID: "4e7b899b-9bd2-4fb6-907c-69e88b59c1b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:21.236551 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:21.236505 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:21.236751 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:21.236661 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:21.236886 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:21.236852 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:21.236993 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:21.236970 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:21.237046 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:21.237015 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:21.237123 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:21.237095 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:23.236477 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:23.236368 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:23.236942 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:23.236529 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:23.236942 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:23.236588 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:23.236942 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:23.236661 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:23.236942 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:23.236785 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:23.236942 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:23.236929 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:24.787824 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:24.787770 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:24.788351 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:24.787917 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:24.788351 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:24.787992 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs podName:0f5b7926-67d9-4e32-bc4e-0c14a42118ea nodeName:}" failed. No retries permitted until 2026-07-07 14:33:32.787969362 +0000 UTC m=+18.063052784 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs") pod "network-metrics-daemon-mzxzk" (UID: "0f5b7926-67d9-4e32-bc4e-0c14a42118ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:24.888928 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:24.888859 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:24.889109 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:24.889035 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:24.889162 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:24.889117 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret podName:91b21080-c287-47e7-9829-601c1be484f2 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:32.889095359 +0000 UTC m=+18.164178780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret") pod "global-pull-secret-syncer-xb5ll" (UID: "91b21080-c287-47e7-9829-601c1be484f2") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:24.989381 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:24.989346 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pn6\" (UniqueName: \"kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6\") pod \"network-check-target-jrxsr\" (UID: \"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4\") " pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:24.989569 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:24.989550 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:33:24.989626 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:24.989576 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:33:24.989626 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:24.989590 2581 projected.go:196] Error preparing data for projected volume kube-api-access-z8pn6 for pod openshift-network-diagnostics/network-check-target-jrxsr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:24.989709 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:24.989661 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6 podName:4e7b899b-9bd2-4fb6-907c-69e88b59c1b4 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:32.989641293 +0000 UTC m=+18.264724715 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8pn6" (UniqueName: "kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6") pod "network-check-target-jrxsr" (UID: "4e7b899b-9bd2-4fb6-907c-69e88b59c1b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:25.237741 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:25.237071 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:25.237741 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:25.237214 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:25.237741 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:25.237609 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:25.237741 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:25.237701 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:25.238164 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:25.237790 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:25.238164 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:25.237937 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:27.237012 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:27.236035 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:27.237012 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:27.236423 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:27.237012 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:27.236848 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:27.237012 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:27.236959 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:27.237925 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:27.237753 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:27.237925 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:27.237866 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:29.239108 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:29.239084 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:29.239478 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:29.239121 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:29.239478 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:29.239192 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:29.239478 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:29.239211 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:29.239478 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:29.239274 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:29.239478 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:29.239381 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:30.300695 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:30.300659 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" event={"ID":"bb461e6a-9c87-4833-8a9e-ef0e6e9c18da","Type":"ContainerStarted","Data":"d2be99d7f4fb10ed5300927dfbac01a5d4203626900ddc1bfa043119462bd7fa"} Jul 07 14:33:30.302239 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:30.302213 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-146.ec2.internal" event={"ID":"00c897cdd111a3fcc4bf6f54f3d25813","Type":"ContainerStarted","Data":"cab034ecb8d1e33ee5221790e3b6a3972950d0ceba08af089a42d7c14d9cc499"} Jul 07 14:33:30.320957 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:30.320907 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l5qx6" podStartSLOduration=2.598490009 podStartE2EDuration="14.320889681s" totalImagesPullingTime="11.722399672s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:30.319879464 +0000 UTC m=+15.594962905" watchObservedRunningTime="2026-07-07 14:33:30.320889681 +0000 UTC m=+15.595973110" Jul 07 14:33:30.334001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:30.333964 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-146.ec2.internal" podStartSLOduration=13.333953517 podStartE2EDuration="13.333953517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:33:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:30.333908193 +0000 UTC m=+15.608991619" watchObservedRunningTime="2026-07-07 14:33:30.333953517 +0000 UTC m=+15.609036943" Jul 07 14:33:31.239676 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.239416 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:31.239861 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.239449 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:31.239861 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.239449 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:31.239861 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:31.239733 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:31.239861 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:31.239831 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:31.240068 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:31.239888 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:31.305673 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.305632 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" event={"ID":"d9df984a94f344c9e5dde59a74f6e184","Type":"ContainerStarted","Data":"b9cef3f763c0ddc81333c9e7e121ec47af83aa3cd7c9f7c75804bbd76f7112ed"} Jul 07 14:33:31.307448 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.307421 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" event={"ID":"09b3e817-b4bd-4864-80f1-835e11bd58f3","Type":"ContainerStarted","Data":"b5ff877a98fa47cc7747cdd26534638f5f90c46a5bce588fb6a69abeb5b9b263"} Jul 07 14:33:31.309020 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.308995 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" event={"ID":"03c12bd0-a596-4c34-977c-3dea4a7d4fe0","Type":"ContainerStarted","Data":"19bfe4d859682351684e28becc71295dbe7b838cfdd23190bfa871109e464fb7"} Jul 07 14:33:31.310582 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.310546 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qlwmh" event={"ID":"2904380c-b1fe-4350-83ce-2c0ec2859ed7","Type":"ContainerStarted","Data":"e1503950a74376ac3fd768bf1c5df1e91d9b8734aaba48c698362cda2e721264"} Jul 07 14:33:31.312098 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.312078 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6hfkd" event={"ID":"17ef7399-7fb7-4446-b7b5-c85cc6d7e43e","Type":"ContainerStarted","Data":"8271308ac16963f1bacf614c31a342188d81afc97669b90728bb48086bab78e8"} Jul 07 14:33:31.313520 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.313492 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-92t44" event={"ID":"9ecd6676-11b6-49ad-b15f-e08fbc43c0b6","Type":"ContainerStarted","Data":"4e1bd76f0c2b5bde047547ce7a7d12bdd98b7fac852e0c519fee167974a2f21f"} Jul 07 14:33:31.314930 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.314907 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fcqkx" event={"ID":"4a097e24-d671-4a5a-96c4-c3f50e38d5e3","Type":"ContainerStarted","Data":"58c40be39442852e337c9c00f12ebac3d2e4f68688dcbf8fbca0dedb1a1ea0fa"} Jul 07 14:33:31.342245 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.342189 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qlwmh" podStartSLOduration=3.619884022 podStartE2EDuration="15.342171184s" totalImagesPullingTime="11.722287162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:31.340742659 +0000 UTC m=+16.615826086" watchObservedRunningTime="2026-07-07 14:33:31.342171184 +0000 UTC m=+16.617254611" Jul 07 14:33:31.360297 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.360248 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-92t44" podStartSLOduration=3.628843147 podStartE2EDuration="15.360234016s" totalImagesPullingTime="11.731390869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:31.359650828 +0000 UTC m=+16.634734279" watchObservedRunningTime="2026-07-07 14:33:31.360234016 +0000 UTC m=+16.635317442" Jul 07 14:33:31.408124 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.408078 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6hfkd" podStartSLOduration=3.760559969 podStartE2EDuration="15.408062495s" totalImagesPullingTime="11.647502526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:31.377955627 +0000 UTC m=+16.653039056" watchObservedRunningTime="2026-07-07 14:33:31.408062495 +0000 UTC m=+16.683145923" Jul 07 14:33:31.437598 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:31.437542 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fcqkx" podStartSLOduration=3.69605515 podStartE2EDuration="15.437522963s" totalImagesPullingTime="11.741467813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:31.437355489 +0000 UTC m=+16.712438916" watchObservedRunningTime="2026-07-07 14:33:31.437522963 +0000 UTC m=+16.712606391" Jul 07 14:33:32.854688 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:32.854651 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:32.855285 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:32.854775 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:32.855285 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:32.854849 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs podName:0f5b7926-67d9-4e32-bc4e-0c14a42118ea nodeName:}" failed. No retries permitted until 2026-07-07 14:33:48.854829806 +0000 UTC m=+34.129913224 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs") pod "network-metrics-daemon-mzxzk" (UID: "0f5b7926-67d9-4e32-bc4e-0c14a42118ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:32.955695 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:32.955656 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:32.955890 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:32.955857 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:32.955942 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:32.955933 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret podName:91b21080-c287-47e7-9829-601c1be484f2 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:48.955914368 +0000 UTC m=+34.230997775 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret") pod "global-pull-secret-syncer-xb5ll" (UID: "91b21080-c287-47e7-9829-601c1be484f2") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:33.056468 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:33.056435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pn6\" (UniqueName: \"kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6\") pod \"network-check-target-jrxsr\" (UID: \"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4\") " pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:33.056643 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:33.056627 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:33:33.056706 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:33.056655 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:33:33.056706 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:33.056668 2581 projected.go:196] Error preparing data for projected volume kube-api-access-z8pn6 for pod openshift-network-diagnostics/network-check-target-jrxsr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:33.056797 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:33.056736 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6 podName:4e7b899b-9bd2-4fb6-907c-69e88b59c1b4 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:49.056713675 +0000 UTC m=+34.331797079 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8pn6" (UniqueName: "kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6") pod "network-check-target-jrxsr" (UID: "4e7b899b-9bd2-4fb6-907c-69e88b59c1b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:33.237024 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:33.236926 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:33.237188 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:33.236915 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:33.237188 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:33.237071 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:33.237308 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:33.237286 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:33.237370 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:33.237338 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:33.237435 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:33.237415 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:33.318244 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:33.318207 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:33.318789 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:33.318768 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:35.237222 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:35.237191 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:35.237871 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:35.237269 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:35.237871 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:35.237279 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:35.237871 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:35.237386 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:35.237871 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:35.237422 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:35.237871 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:35.237501 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:35.911246 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:35.911212 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:35.911432 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:35.911333 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 14:33:35.912062 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:35.912037 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fcqkx" Jul 07 14:33:37.236614 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:37.236579 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:37.237185 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:37.236627 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:37.237185 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:37.236693 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:37.237185 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:37.236716 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:37.237185 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:37.236787 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:37.237185 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:37.236909 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:39.236344 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.236038 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:39.236811 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.236049 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:39.236811 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:39.236420 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:39.236811 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.236121 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:39.236811 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:39.236501 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:39.236811 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:39.236598 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:39.332347 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.332314 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" event={"ID":"ea46ddfe-55c0-47f0-8a9a-911f295bf044","Type":"ContainerStarted","Data":"076e7b9e85bbc5e0425db54fb87a0fba71bf7e8bf192dd7ea5943ed0d5e88a66"} Jul 07 14:33:39.332511 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.332389 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" event={"ID":"ea46ddfe-55c0-47f0-8a9a-911f295bf044","Type":"ContainerStarted","Data":"4145e73711300ad3a113801bfe1d16ed5e2f582c44b6562a06ebd4570d3b5706"} Jul 07 14:33:39.332511 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.332437 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" event={"ID":"ea46ddfe-55c0-47f0-8a9a-911f295bf044","Type":"ContainerStarted","Data":"60ece521267e9b2247aba7919031481bbf4d8620b2fe119a90f7ae9e16340859"} Jul 07 14:33:39.332511 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.332475 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" event={"ID":"ea46ddfe-55c0-47f0-8a9a-911f295bf044","Type":"ContainerStarted","Data":"7ee78338472d0273457b884bd1996cff0a94969242da86fc512244557b1ca701"} Jul 07 14:33:39.332511 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.332504 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" event={"ID":"ea46ddfe-55c0-47f0-8a9a-911f295bf044","Type":"ContainerStarted","Data":"6d44cc572a18367022a9eed56e0845798bd5402d250acfedbdf10c64bf099c5a"} Jul 07 14:33:39.332682 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.332546 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" event={"ID":"ea46ddfe-55c0-47f0-8a9a-911f295bf044","Type":"ContainerStarted","Data":"6e114c043350665b2ddd410642ad283db9457fea6840183147ff09ade8213d9c"} Jul 07 14:33:39.333734 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.333711 2581 generic.go:356] "Generic (PLEG): container finished" podID="03c12bd0-a596-4c34-977c-3dea4a7d4fe0" containerID="19bfe4d859682351684e28becc71295dbe7b838cfdd23190bfa871109e464fb7" exitCode=0 Jul 07 14:33:39.333842 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.333781 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" event={"ID":"03c12bd0-a596-4c34-977c-3dea4a7d4fe0","Type":"ContainerDied","Data":"19bfe4d859682351684e28becc71295dbe7b838cfdd23190bfa871109e464fb7"} Jul 07 14:33:39.335309 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.335288 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x2bcs" event={"ID":"b1cea48d-ba43-4419-9966-bc2b80b2fcc0","Type":"ContainerStarted","Data":"2d1f06e83c0fe35f4844c84140e7839d20d02b8ca2fb3ff750703964551c7d82"} Jul 07 14:33:39.336645 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.336626 2581 generic.go:356] "Generic (PLEG): container finished" podID="d9df984a94f344c9e5dde59a74f6e184" containerID="b9cef3f763c0ddc81333c9e7e121ec47af83aa3cd7c9f7c75804bbd76f7112ed" exitCode=0 Jul 07 14:33:39.336746 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.336653 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" event={"ID":"d9df984a94f344c9e5dde59a74f6e184","Type":"ContainerDied","Data":"b9cef3f763c0ddc81333c9e7e121ec47af83aa3cd7c9f7c75804bbd76f7112ed"} Jul 07 14:33:39.388791 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.388746 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x2bcs" podStartSLOduration=2.381398076 podStartE2EDuration="23.388729975s" totalImagesPullingTime="21.007331899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:39.38853023 +0000 UTC m=+24.663613657" watchObservedRunningTime="2026-07-07 14:33:39.388729975 +0000 UTC m=+24.663813400" Jul 07 14:33:39.613590 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:39.613562 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 14:33:40.316308 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:40.316206 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T14:33:39.613576505Z","UUID":"6aafc7fe-4621-4aad-a3cb-c781bbd8a371","Handler":null,"Name":"","Endpoint":""} Jul 07 14:33:40.317906 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:40.317884 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 14:33:40.318036 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:40.317929 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 14:33:40.340957 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:40.340930 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" event={"ID":"d9df984a94f344c9e5dde59a74f6e184","Type":"ContainerStarted","Data":"5d12934ff327b171bf95b5757300482f0d379f6e75049a08fe7ecacb083b1015"} Jul 07 14:33:40.342543 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:40.342523 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" event={"ID":"09b3e817-b4bd-4864-80f1-835e11bd58f3","Type":"ContainerStarted","Data":"1e9cba4a2379238347b84627cebf7bdef8e1b11af39916c8c25d2e14e728183f"} Jul 07 14:33:40.357538 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:40.357495 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-146.ec2.internal" podStartSLOduration=23.357481063 podStartE2EDuration="23.357481063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:33:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:40.357345976 +0000 UTC m=+25.632429402" watchObservedRunningTime="2026-07-07 14:33:40.357481063 +0000 UTC m=+25.632564489" Jul 07 14:33:41.236631 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:41.236543 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:41.236800 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:41.236655 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:41.236800 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:41.236546 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:41.236800 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:41.236721 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:41.236800 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:41.236673 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:41.236800 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:41.236770 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:41.348908 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:41.348865 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" event={"ID":"ea46ddfe-55c0-47f0-8a9a-911f295bf044","Type":"ContainerStarted","Data":"05c1d48753e04036df2784df18772c43423dc020496e98a358bfd1bbf11478fe"} Jul 07 14:33:41.351114 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:41.351090 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" event={"ID":"09b3e817-b4bd-4864-80f1-835e11bd58f3","Type":"ContainerStarted","Data":"bc270668112e96b08474bd12e70e33bce8bc84a9c1b4bedc5fdb4844af023384"} Jul 07 14:33:41.370548 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:41.370504 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8sqmk" podStartSLOduration=3.898081065 podStartE2EDuration="25.370489928s" totalImagesPullingTime="21.472408863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:41.37041614 +0000 UTC m=+26.645499567" watchObservedRunningTime="2026-07-07 14:33:41.370489928 +0000 UTC m=+26.645573353" Jul 07 14:33:43.236601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:43.236144 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:43.236601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:43.236144 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:43.236601 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:43.236289 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:43.236601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:43.236302 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:43.236601 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:43.236387 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:43.236601 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:43.236468 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:43.359330 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:43.359125 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" event={"ID":"ea46ddfe-55c0-47f0-8a9a-911f295bf044","Type":"ContainerStarted","Data":"44a3586cd282bbb68948e5588daeae5edc401871911ac8f015c6615ee16e5a3d"} Jul 07 14:33:43.359511 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:43.359460 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:43.374641 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:43.374619 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:43.388289 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:43.388246 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" podStartSLOduration=6.43264322 podStartE2EDuration="27.388233306s" totalImagesPullingTime="20.955590086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:43.386422148 +0000 UTC m=+28.661505573" watchObservedRunningTime="2026-07-07 14:33:43.388233306 +0000 UTC m=+28.663316731" Jul 07 14:33:44.363041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:44.362902 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:44.363041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:44.362986 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:44.382235 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:44.382208 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:33:45.237994 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:45.237944 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:45.238161 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:45.238036 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:45.238161 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:45.238088 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:45.238161 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:45.238103 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:45.238333 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:45.238168 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:45.238333 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:45.238264 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:45.401708 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:45.401675 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mzxzk"] Jul 07 14:33:45.402239 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:45.401798 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:45.402239 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:45.401935 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:45.402489 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:45.402466 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xb5ll"] Jul 07 14:33:45.402614 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:45.402532 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:45.402614 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:45.402599 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:45.403723 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:45.403700 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jrxsr"] Jul 07 14:33:45.403845 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:45.403796 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:45.403929 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:45.403909 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:46.367446 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:46.367417 2581 generic.go:356] "Generic (PLEG): container finished" podID="03c12bd0-a596-4c34-977c-3dea4a7d4fe0" containerID="42ce7468f5c7f3a3fe4a6c010375ccab8b5f91de241f81414d88d22199cddd9a" exitCode=0 Jul 07 14:33:46.367579 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:46.367491 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" event={"ID":"03c12bd0-a596-4c34-977c-3dea4a7d4fe0","Type":"ContainerDied","Data":"42ce7468f5c7f3a3fe4a6c010375ccab8b5f91de241f81414d88d22199cddd9a"} Jul 07 14:33:47.238494 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:47.237970 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:47.238494 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:47.237986 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:47.238494 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:47.238106 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:47.238494 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:47.238176 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:47.238494 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:47.238246 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:47.238494 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:47.238342 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:48.373673 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:48.373643 2581 generic.go:356] "Generic (PLEG): container finished" podID="03c12bd0-a596-4c34-977c-3dea4a7d4fe0" containerID="071c19966351a5b685ba5d61684de56a89776b56d94c17a7e4d0221bb544dbff" exitCode=0 Jul 07 14:33:48.374171 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:48.373687 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" event={"ID":"03c12bd0-a596-4c34-977c-3dea4a7d4fe0","Type":"ContainerDied","Data":"071c19966351a5b685ba5d61684de56a89776b56d94c17a7e4d0221bb544dbff"} Jul 07 14:33:48.870518 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:48.870486 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:48.870675 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:48.870631 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:48.870711 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:48.870690 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs podName:0f5b7926-67d9-4e32-bc4e-0c14a42118ea nodeName:}" failed. No retries permitted until 2026-07-07 14:34:20.870675821 +0000 UTC m=+66.145759230 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs") pod "network-metrics-daemon-mzxzk" (UID: "0f5b7926-67d9-4e32-bc4e-0c14a42118ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:33:48.971096 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:48.971063 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:48.971239 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:48.971218 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:48.971298 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:48.971289 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret podName:91b21080-c287-47e7-9829-601c1be484f2 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:20.971271481 +0000 UTC m=+66.246354886 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret") pod "global-pull-secret-syncer-xb5ll" (UID: "91b21080-c287-47e7-9829-601c1be484f2") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:33:49.072232 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:49.072194 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pn6\" (UniqueName: \"kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6\") pod \"network-check-target-jrxsr\" (UID: \"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4\") " pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:49.072389 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:49.072357 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:33:49.072389 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:49.072377 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:33:49.072389 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:49.072387 2581 projected.go:196] Error preparing data for projected volume kube-api-access-z8pn6 for pod openshift-network-diagnostics/network-check-target-jrxsr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:49.072476 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:49.072439 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6 podName:4e7b899b-9bd2-4fb6-907c-69e88b59c1b4 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:21.072425607 +0000 UTC m=+66.347509011 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8pn6" (UniqueName: "kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6") pod "network-check-target-jrxsr" (UID: "4e7b899b-9bd2-4fb6-907c-69e88b59c1b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:33:49.235885 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:49.235796 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:49.236064 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:49.235797 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:49.236064 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:49.235936 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:49.236064 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:49.235937 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mzxzk" podUID="0f5b7926-67d9-4e32-bc4e-0c14a42118ea" Jul 07 14:33:49.236064 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:49.236005 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xb5ll" podUID="91b21080-c287-47e7-9829-601c1be484f2" Jul 07 14:33:49.236256 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:49.236087 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jrxsr" podUID="4e7b899b-9bd2-4fb6-907c-69e88b59c1b4" Jul 07 14:33:50.380939 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:50.380904 2581 generic.go:356] "Generic (PLEG): container finished" podID="03c12bd0-a596-4c34-977c-3dea4a7d4fe0" containerID="e5a62cc81da13cf13646e6a7fa80b15447a6a5cf2f315b985e30e63675026d72" exitCode=0 Jul 07 14:33:50.381617 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:50.380943 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" event={"ID":"03c12bd0-a596-4c34-977c-3dea4a7d4fe0","Type":"ContainerDied","Data":"e5a62cc81da13cf13646e6a7fa80b15447a6a5cf2f315b985e30e63675026d72"} Jul 07 14:33:51.003292 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.003267 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-146.ec2.internal" event="NodeReady" Jul 07 14:33:51.003470 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.003393 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 14:33:51.047627 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.047596 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56b97d49f8-nsl7p"] Jul 07 14:33:51.066117 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.066047 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-gwq7g"] Jul 07 14:33:51.066277 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.066176 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.073774 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.073751 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 14:33:51.073940 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.073869 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 14:33:51.074055 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.074032 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qgdmq\"" Jul 07 14:33:51.076693 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.076598 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2"] Jul 07 14:33:51.076896 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.076872 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.079411 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.079396 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 14:33:51.081714 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.081686 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 14:33:51.081836 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.081797 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 14:33:51.081940 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.081924 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 14:33:51.082244 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.082228 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5qp2d\"" Jul 07 14:33:51.083623 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.083501 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 14:33:51.089078 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.089055 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq"] Jul 07 14:33:51.089185 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.089143 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:51.091522 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.091505 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 14:33:51.091655 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.091505 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 14:33:51.091932 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.091913 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 14:33:51.093293 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.093273 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 14:33:51.093502 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.093336 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 14:33:51.093602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.093588 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ncfp7\"" Jul 07 14:33:51.093602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.093596 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 14:33:51.101524 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.101499 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8"] Jul 07 14:33:51.101693 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.101674 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:33:51.105121 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.105014 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 14:33:51.105121 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.105044 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:33:51.105593 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.105568 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 14:33:51.109500 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.109485 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-45kw9\"" Jul 07 14:33:51.114124 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.114104 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-67bf4497f9-krpmz"] Jul 07 14:33:51.114287 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.114246 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" Jul 07 14:33:51.116785 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.116765 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:33:51.116905 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.116890 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-sx84z\"" Jul 07 14:33:51.117139 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.117123 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 14:33:51.117509 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.117249 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 14:33:51.117509 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.117340 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 14:33:51.123094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.123064 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-wdldm"] Jul 07 14:33:51.123206 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.123191 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.125630 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.125611 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 14:33:51.125741 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.125689 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 14:33:51.125798 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.125770 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 14:33:51.126633 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.126614 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 14:33:51.126739 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.126695 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 14:33:51.126806 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.126737 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 14:33:51.126891 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.126861 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-75lw4\"" Jul 07 14:33:51.139415 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.139191 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl"] Jul 07 14:33:51.139531 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.139506 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-wdldm" Jul 07 14:33:51.141885 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.141866 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 14:33:51.141963 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.141872 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:33:51.141963 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.141937 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-25shp\"" Jul 07 14:33:51.149655 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.149633 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-6rwzn"] Jul 07 14:33:51.149770 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.149757 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" Jul 07 14:33:51.152404 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.152384 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 14:33:51.152495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.152468 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 14:33:51.152746 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.152728 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:33:51.152793 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.152728 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9ck5f\"" Jul 07 14:33:51.153214 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.153199 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 14:33:51.163966 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.163948 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-hkxn9"] Jul 07 14:33:51.164090 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.164076 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.167393 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.167375 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 14:33:51.167484 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.167380 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:33:51.167484 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.167453 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-f4sdj\"" Jul 07 14:33:51.167842 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.167827 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 14:33:51.167898 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.167883 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 14:33:51.173971 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.173954 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 14:33:51.174563 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.174545 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2dz7j"] Jul 07 14:33:51.174698 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.174665 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-hkxn9" Jul 07 14:33:51.177328 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.177313 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 14:33:51.177414 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.177353 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-js2fn\"" Jul 07 14:33:51.178172 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.178156 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 14:33:51.188853 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.188835 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2vgh2"] Jul 07 14:33:51.188951 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.188900 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.189928 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.189908 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.190026 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.189947 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2nq6\" (UniqueName: \"kubernetes.io/projected/ffc93eac-e8ab-406a-96df-22c97fec6622-kube-api-access-k2nq6\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:51.190026 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190003 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-serving-cert\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.190026 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190023 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-certificates\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.190185 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190042 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93cb5a02-4235-47be-aedc-a5d73c3d117e-ca-trust-extracted\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.190185 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190059 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:33:51.190185 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190103 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.190185 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190124 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7bgw\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-kube-api-access-r7bgw\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.190185 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190141 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-installation-pull-secrets\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.190185 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190171 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-snapshots\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.190470 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.190470 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190203 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-bound-sa-token\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.190470 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190222 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-image-registry-private-configuration\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.190470 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190250 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-trusted-ca\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.190470 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190279 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ffc93eac-e8ab-406a-96df-22c97fec6622-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:51.190470 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190295 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:51.190470 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190317 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-tmp\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.190470 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190342 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqc92\" (UniqueName: \"kubernetes.io/projected/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-kube-api-access-jqc92\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.190470 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.190379 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vqbc\" (UniqueName: \"kubernetes.io/projected/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-kube-api-access-9vqbc\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:33:51.191644 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.191626 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bxlxt\"" Jul 07 14:33:51.191737 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.191681 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 14:33:51.191870 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.191781 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 14:33:51.199027 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199010 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56b97d49f8-nsl7p"] Jul 07 14:33:51.199104 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199037 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67bf4497f9-krpmz"] Jul 07 14:33:51.199104 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199050 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2"] Jul 07 14:33:51.199104 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199064 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8"] Jul 07 14:33:51.199104 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199076 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl"] Jul 07 14:33:51.199104 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199088 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-gwq7g"] Jul 07 14:33:51.199104 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199099 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-wdldm"] Jul 07 14:33:51.199331 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199112 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2vgh2"] Jul 07 14:33:51.199331 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199112 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2vgh2" Jul 07 14:33:51.199331 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199124 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-6rwzn"] Jul 07 14:33:51.199331 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199135 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq"] Jul 07 14:33:51.199331 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199151 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-hkxn9"] Jul 07 14:33:51.199331 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.199163 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2dz7j"] Jul 07 14:33:51.201343 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.201328 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 14:33:51.201428 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.201368 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kwwz6\"" Jul 07 14:33:51.201428 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.201397 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 14:33:51.201651 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.201636 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 14:33:51.239537 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.239513 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:33:51.239678 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.239601 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:33:51.239678 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.239627 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:33:51.241901 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.241882 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fmsjq\"" Jul 07 14:33:51.242088 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.242075 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 14:33:51.242139 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.242116 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 14:33:51.242530 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.242515 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bf67t\"" Jul 07 14:33:51.310529 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.308450 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-snapshots\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.310529 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.309989 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.310767 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310568 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.310767 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310609 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-bound-sa-token\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.310767 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310640 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5g9j\" (UniqueName: \"kubernetes.io/projected/adbb655f-b138-4258-909d-f74f718863a1-kube-api-access-m5g9j\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.310767 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310667 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.310767 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310697 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktnfd\" (UniqueName: \"kubernetes.io/projected/7982337d-d41b-40b6-ab24-d925e4ac6196-kube-api-access-ktnfd\") pod \"ingress-canary-2vgh2\" (UID: \"7982337d-d41b-40b6-ab24-d925e4ac6196\") " pod="openshift-ingress-canary/ingress-canary-2vgh2" Jul 07 14:33:51.310767 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310725 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:51.310767 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310754 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vqbc\" (UniqueName: \"kubernetes.io/projected/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-kube-api-access-9vqbc\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:33:51.311094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310779 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/adbb655f-b138-4258-909d-f74f718863a1-config-volume\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.311094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310807 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nbtn\" (UniqueName: \"kubernetes.io/projected/615e9275-9e38-4727-8c1f-98b3e94b86e1-kube-api-access-4nbtn\") pod \"kube-storage-version-migrator-operator-846bf668f7-d5kgl\" (UID: \"615e9275-9e38-4727-8c1f-98b3e94b86e1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" Jul 07 14:33:51.311094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310858 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8dg4\" (UniqueName: \"kubernetes.io/projected/88545de4-3bf9-41ba-8e60-03d91dc8bd10-kube-api-access-z8dg4\") pod \"volume-data-source-validator-5d456fdbcd-wdldm\" (UID: \"88545de4-3bf9-41ba-8e60-03d91dc8bd10\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-wdldm" Jul 07 14:33:51.311094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310902 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2nq6\" (UniqueName: \"kubernetes.io/projected/ffc93eac-e8ab-406a-96df-22c97fec6622-kube-api-access-k2nq6\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:51.311094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.310927 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbkbk\" (UniqueName: \"kubernetes.io/projected/1fc8f4f8-d97a-4bb0-b29a-369b649e1a34-kube-api-access-tbkbk\") pod \"network-check-source-989f8b8dc-hkxn9\" (UID: \"1fc8f4f8-d97a-4bb0-b29a-369b649e1a34\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-hkxn9" Jul 07 14:33:51.312839 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.312475 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/615e9275-9e38-4727-8c1f-98b3e94b86e1-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-d5kgl\" (UID: \"615e9275-9e38-4727-8c1f-98b3e94b86e1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" Jul 07 14:33:51.312839 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.312543 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kvcm\" (UniqueName: \"kubernetes.io/projected/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-kube-api-access-4kvcm\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.312839 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.312629 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-snapshots\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.312839 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.312755 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-installation-pull-secrets\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.313129 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.312971 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:33:51.313129 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.313057 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls podName:ffc93eac-e8ab-406a-96df-22c97fec6622 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:51.813034308 +0000 UTC m=+37.088117716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4c5x2" (UID: "ffc93eac-e8ab-406a-96df-22c97fec6622") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:33:51.313225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313156 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27bf9b12-515c-49fd-902c-9ca037f5fa38-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-lqck8\" (UID: \"27bf9b12-515c-49fd-902c-9ca037f5fa38\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" Jul 07 14:33:51.313225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313203 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/615e9275-9e38-4727-8c1f-98b3e94b86e1-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-d5kgl\" (UID: \"615e9275-9e38-4727-8c1f-98b3e94b86e1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" Jul 07 14:33:51.313316 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-image-registry-private-configuration\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.313438 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313278 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-trusted-ca\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.313438 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/88bef153-2452-4b8c-84f4-333d8426694d-config\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313625 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ffc93eac-e8ab-406a-96df-22c97fec6622-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313680 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqc92\" (UniqueName: \"kubernetes.io/projected/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-kube-api-access-jqc92\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313725 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-tmp\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313768 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313799 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6jmz\" (UniqueName: \"kubernetes.io/projected/88bef153-2452-4b8c-84f4-333d8426694d-kube-api-access-l6jmz\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313863 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313892 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-stats-auth\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313933 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88bef153-2452-4b8c-84f4-333d8426694d-trusted-ca\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.313963 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7982337d-d41b-40b6-ab24-d925e4ac6196-cert\") pod \"ingress-canary-2vgh2\" (UID: \"7982337d-d41b-40b6-ab24-d925e4ac6196\") " pod="openshift-ingress-canary/ingress-canary-2vgh2" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-default-certificate\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-serving-cert\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314096 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-certificates\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314119 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314162 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adbb655f-b138-4258-909d-f74f718863a1-tmp-dir\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314197 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93cb5a02-4235-47be-aedc-a5d73c3d117e-ca-trust-extracted\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.314495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314240 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/88bef153-2452-4b8c-84f4-333d8426694d-serving-cert\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.315265 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314307 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:33:51.315265 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7bgw\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-kube-api-access-r7bgw\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.315265 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314383 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27bf9b12-515c-49fd-902c-9ca037f5fa38-config\") pod \"service-ca-operator-5d46d9ffd4-lqck8\" (UID: \"27bf9b12-515c-49fd-902c-9ca037f5fa38\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" Jul 07 14:33:51.315265 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314417 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.315265 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.314461 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5wtt\" (UniqueName: \"kubernetes.io/projected/27bf9b12-515c-49fd-902c-9ca037f5fa38-kube-api-access-d5wtt\") pod \"service-ca-operator-5d46d9ffd4-lqck8\" (UID: \"27bf9b12-515c-49fd-902c-9ca037f5fa38\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" Jul 07 14:33:51.315937 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.315789 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93cb5a02-4235-47be-aedc-a5d73c3d117e-ca-trust-extracted\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.316768 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.316418 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.316768 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.316430 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-certificates\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.316768 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.316496 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:33:51.316768 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.316499 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:33:51.316768 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.316518 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b97d49f8-nsl7p: secret "image-registry-tls" not found Jul 07 14:33:51.316768 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.316609 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls podName:fada1bb6-9ff9-402c-b807-0a6a081e6dd0 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:51.816590202 +0000 UTC m=+37.091673620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zbvfq" (UID: "fada1bb6-9ff9-402c-b807-0a6a081e6dd0") : secret "samples-operator-tls" not found Jul 07 14:33:51.317142 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.317043 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-tmp\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.317198 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.317176 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ffc93eac-e8ab-406a-96df-22c97fec6622-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:51.318373 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.317667 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls podName:93cb5a02-4235-47be-aedc-a5d73c3d117e nodeName:}" failed. No retries permitted until 2026-07-07 14:33:51.817646852 +0000 UTC m=+37.092730271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls") pod "image-registry-56b97d49f8-nsl7p" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e") : secret "image-registry-tls" not found Jul 07 14:33:51.318373 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.317804 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-trusted-ca\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.319195 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.319169 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-installation-pull-secrets\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.319630 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.319606 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-image-registry-private-configuration\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.320200 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.320184 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-serving-cert\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.324226 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.324207 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vqbc\" (UniqueName: \"kubernetes.io/projected/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-kube-api-access-9vqbc\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:33:51.327630 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.327604 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-bound-sa-token\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.329121 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.329085 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqc92\" (UniqueName: \"kubernetes.io/projected/dd9bcdd4-6d7b-4809-befd-7238b2ceb035-kube-api-access-jqc92\") pod \"insights-operator-5c847dcfc4-gwq7g\" (UID: \"dd9bcdd4-6d7b-4809-befd-7238b2ceb035\") " pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.329289 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.329274 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7bgw\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-kube-api-access-r7bgw\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.329508 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.329491 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2nq6\" (UniqueName: \"kubernetes.io/projected/ffc93eac-e8ab-406a-96df-22c97fec6622-kube-api-access-k2nq6\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:51.388252 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.388225 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" Jul 07 14:33:51.415888 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.415862 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27bf9b12-515c-49fd-902c-9ca037f5fa38-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-lqck8\" (UID: \"27bf9b12-515c-49fd-902c-9ca037f5fa38\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" Jul 07 14:33:51.416019 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.415900 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/615e9275-9e38-4727-8c1f-98b3e94b86e1-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-d5kgl\" (UID: \"615e9275-9e38-4727-8c1f-98b3e94b86e1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" Jul 07 14:33:51.416019 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.415922 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/88bef153-2452-4b8c-84f4-333d8426694d-config\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.416145 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.416123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.416201 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.416171 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6jmz\" (UniqueName: \"kubernetes.io/projected/88bef153-2452-4b8c-84f4-333d8426694d-kube-api-access-l6jmz\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.416258 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.416229 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-stats-auth\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.416332 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.416254 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88bef153-2452-4b8c-84f4-333d8426694d-trusted-ca\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.416332 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.416250 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:33:51.416430 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.416363 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:33:51.916340569 +0000 UTC m=+37.191423994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : secret "router-metrics-certs-default" not found Jul 07 14:33:51.416487 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.416280 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7982337d-d41b-40b6-ab24-d925e4ac6196-cert\") pod \"ingress-canary-2vgh2\" (UID: \"7982337d-d41b-40b6-ab24-d925e4ac6196\") " pod="openshift-ingress-canary/ingress-canary-2vgh2" Jul 07 14:33:51.416487 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.416472 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-default-certificate\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.416589 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.416513 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.416589 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.416536 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adbb655f-b138-4258-909d-f74f718863a1-tmp-dir\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.416589 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.416567 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/88bef153-2452-4b8c-84f4-333d8426694d-serving-cert\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.416742 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.416608 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27bf9b12-515c-49fd-902c-9ca037f5fa38-config\") pod \"service-ca-operator-5d46d9ffd4-lqck8\" (UID: \"27bf9b12-515c-49fd-902c-9ca037f5fa38\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" Jul 07 14:33:51.417299 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.416682 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:33:51.417405 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.417370 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls podName:adbb655f-b138-4258-909d-f74f718863a1 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:51.917353303 +0000 UTC m=+37.192436722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls") pod "dns-default-2dz7j" (UID: "adbb655f-b138-4258-909d-f74f718863a1") : secret "dns-default-metrics-tls" not found Jul 07 14:33:51.417405 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.417034 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adbb655f-b138-4258-909d-f74f718863a1-tmp-dir\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.417405 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.416920 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/88bef153-2452-4b8c-84f4-333d8426694d-config\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.417580 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.417181 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27bf9b12-515c-49fd-902c-9ca037f5fa38-config\") pod \"service-ca-operator-5d46d9ffd4-lqck8\" (UID: \"27bf9b12-515c-49fd-902c-9ca037f5fa38\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" Jul 07 14:33:51.417580 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.417066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5wtt\" (UniqueName: \"kubernetes.io/projected/27bf9b12-515c-49fd-902c-9ca037f5fa38-kube-api-access-d5wtt\") pod \"service-ca-operator-5d46d9ffd4-lqck8\" (UID: \"27bf9b12-515c-49fd-902c-9ca037f5fa38\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" Jul 07 14:33:51.417580 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.417493 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5g9j\" (UniqueName: \"kubernetes.io/projected/adbb655f-b138-4258-909d-f74f718863a1-kube-api-access-m5g9j\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.417580 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.417286 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88bef153-2452-4b8c-84f4-333d8426694d-trusted-ca\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.417844 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.417707 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:33:51.917689737 +0000 UTC m=+37.192773143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : configmap references non-existent config key: service-ca.crt Jul 07 14:33:51.417844 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.417798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.417957 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.417878 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktnfd\" (UniqueName: \"kubernetes.io/projected/7982337d-d41b-40b6-ab24-d925e4ac6196-kube-api-access-ktnfd\") pod \"ingress-canary-2vgh2\" (UID: \"7982337d-d41b-40b6-ab24-d925e4ac6196\") " pod="openshift-ingress-canary/ingress-canary-2vgh2" Jul 07 14:33:51.417957 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.417930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/adbb655f-b138-4258-909d-f74f718863a1-config-volume\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.418141 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.418124 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nbtn\" (UniqueName: \"kubernetes.io/projected/615e9275-9e38-4727-8c1f-98b3e94b86e1-kube-api-access-4nbtn\") pod \"kube-storage-version-migrator-operator-846bf668f7-d5kgl\" (UID: \"615e9275-9e38-4727-8c1f-98b3e94b86e1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" Jul 07 14:33:51.418259 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.418241 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8dg4\" (UniqueName: \"kubernetes.io/projected/88545de4-3bf9-41ba-8e60-03d91dc8bd10-kube-api-access-z8dg4\") pod \"volume-data-source-validator-5d456fdbcd-wdldm\" (UID: \"88545de4-3bf9-41ba-8e60-03d91dc8bd10\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-wdldm" Jul 07 14:33:51.418423 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.418402 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/adbb655f-b138-4258-909d-f74f718863a1-config-volume\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.418494 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.418406 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbkbk\" (UniqueName: \"kubernetes.io/projected/1fc8f4f8-d97a-4bb0-b29a-369b649e1a34-kube-api-access-tbkbk\") pod \"network-check-source-989f8b8dc-hkxn9\" (UID: \"1fc8f4f8-d97a-4bb0-b29a-369b649e1a34\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-hkxn9" Jul 07 14:33:51.418558 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.418488 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/615e9275-9e38-4727-8c1f-98b3e94b86e1-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-d5kgl\" (UID: \"615e9275-9e38-4727-8c1f-98b3e94b86e1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" Jul 07 14:33:51.419246 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.419173 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/615e9275-9e38-4727-8c1f-98b3e94b86e1-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-d5kgl\" (UID: \"615e9275-9e38-4727-8c1f-98b3e94b86e1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" Jul 07 14:33:51.419246 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.419183 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7982337d-d41b-40b6-ab24-d925e4ac6196-cert\") pod \"ingress-canary-2vgh2\" (UID: \"7982337d-d41b-40b6-ab24-d925e4ac6196\") " pod="openshift-ingress-canary/ingress-canary-2vgh2" Jul 07 14:33:51.419769 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.419398 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/615e9275-9e38-4727-8c1f-98b3e94b86e1-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-d5kgl\" (UID: \"615e9275-9e38-4727-8c1f-98b3e94b86e1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" Jul 07 14:33:51.419769 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.418548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kvcm\" (UniqueName: \"kubernetes.io/projected/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-kube-api-access-4kvcm\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.419769 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.419738 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-stats-auth\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.419935 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.419853 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/88bef153-2452-4b8c-84f4-333d8426694d-serving-cert\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.420100 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.420078 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-default-certificate\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.420356 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.420340 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27bf9b12-515c-49fd-902c-9ca037f5fa38-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-lqck8\" (UID: \"27bf9b12-515c-49fd-902c-9ca037f5fa38\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" Jul 07 14:33:51.425474 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.425423 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6jmz\" (UniqueName: \"kubernetes.io/projected/88bef153-2452-4b8c-84f4-333d8426694d-kube-api-access-l6jmz\") pod \"console-operator-68b757fb67-6rwzn\" (UID: \"88bef153-2452-4b8c-84f4-333d8426694d\") " pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.425964 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.425838 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5wtt\" (UniqueName: \"kubernetes.io/projected/27bf9b12-515c-49fd-902c-9ca037f5fa38-kube-api-access-d5wtt\") pod \"service-ca-operator-5d46d9ffd4-lqck8\" (UID: \"27bf9b12-515c-49fd-902c-9ca037f5fa38\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" Jul 07 14:33:51.426752 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.426727 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5g9j\" (UniqueName: \"kubernetes.io/projected/adbb655f-b138-4258-909d-f74f718863a1-kube-api-access-m5g9j\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.427401 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.427377 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbkbk\" (UniqueName: \"kubernetes.io/projected/1fc8f4f8-d97a-4bb0-b29a-369b649e1a34-kube-api-access-tbkbk\") pod \"network-check-source-989f8b8dc-hkxn9\" (UID: \"1fc8f4f8-d97a-4bb0-b29a-369b649e1a34\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-hkxn9" Jul 07 14:33:51.427531 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.427506 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nbtn\" (UniqueName: \"kubernetes.io/projected/615e9275-9e38-4727-8c1f-98b3e94b86e1-kube-api-access-4nbtn\") pod \"kube-storage-version-migrator-operator-846bf668f7-d5kgl\" (UID: \"615e9275-9e38-4727-8c1f-98b3e94b86e1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" Jul 07 14:33:51.427614 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.427598 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kvcm\" (UniqueName: \"kubernetes.io/projected/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-kube-api-access-4kvcm\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.428547 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.428523 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktnfd\" (UniqueName: \"kubernetes.io/projected/7982337d-d41b-40b6-ab24-d925e4ac6196-kube-api-access-ktnfd\") pod \"ingress-canary-2vgh2\" (UID: \"7982337d-d41b-40b6-ab24-d925e4ac6196\") " pod="openshift-ingress-canary/ingress-canary-2vgh2" Jul 07 14:33:51.428721 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.428704 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8dg4\" (UniqueName: \"kubernetes.io/projected/88545de4-3bf9-41ba-8e60-03d91dc8bd10-kube-api-access-z8dg4\") pod \"volume-data-source-validator-5d456fdbcd-wdldm\" (UID: \"88545de4-3bf9-41ba-8e60-03d91dc8bd10\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-wdldm" Jul 07 14:33:51.450799 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.450762 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-wdldm" Jul 07 14:33:51.456888 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.456864 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" Jul 07 14:33:51.472186 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.472008 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:33:51.483082 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.483062 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-hkxn9" Jul 07 14:33:51.507746 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.507712 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2vgh2" Jul 07 14:33:51.603400 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.603272 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-gwq7g"] Jul 07 14:33:51.621046 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:51.618405 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd9bcdd4_6d7b_4809_befd_7238b2ceb035.slice/crio-7a831e393985ca67cbc58eb33c16d510c1bdcb9cbf63d762b766c084eee8c08e WatchSource:0}: Error finding container 7a831e393985ca67cbc58eb33c16d510c1bdcb9cbf63d762b766c084eee8c08e: Status 404 returned error can't find the container with id 7a831e393985ca67cbc58eb33c16d510c1bdcb9cbf63d762b766c084eee8c08e Jul 07 14:33:51.663377 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.663343 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-wdldm"] Jul 07 14:33:51.667130 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:51.667088 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88545de4_3bf9_41ba_8e60_03d91dc8bd10.slice/crio-4b186b056a42283209549c33421e87047ff5c149de30db78791580a50264cb5e WatchSource:0}: Error finding container 4b186b056a42283209549c33421e87047ff5c149de30db78791580a50264cb5e: Status 404 returned error can't find the container with id 4b186b056a42283209549c33421e87047ff5c149de30db78791580a50264cb5e Jul 07 14:33:51.685414 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.685388 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl"] Jul 07 14:33:51.689745 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:51.689715 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615e9275_9e38_4727_8c1f_98b3e94b86e1.slice/crio-1d12a8176d36613fb05f404b2a6ca15dc6e688f83df9246a6fc7359ca36acb26 WatchSource:0}: Error finding container 1d12a8176d36613fb05f404b2a6ca15dc6e688f83df9246a6fc7359ca36acb26: Status 404 returned error can't find the container with id 1d12a8176d36613fb05f404b2a6ca15dc6e688f83df9246a6fc7359ca36acb26 Jul 07 14:33:51.700628 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.700604 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-hkxn9"] Jul 07 14:33:51.703812 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:51.703784 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc8f4f8_d97a_4bb0_b29a_369b649e1a34.slice/crio-c45ca228a9515c56581a635d6bd12bb189b456de094ff16fd028ca3274d0ffbc WatchSource:0}: Error finding container c45ca228a9515c56581a635d6bd12bb189b456de094ff16fd028ca3274d0ffbc: Status 404 returned error can't find the container with id c45ca228a9515c56581a635d6bd12bb189b456de094ff16fd028ca3274d0ffbc Jul 07 14:33:51.715513 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.715446 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-6rwzn"] Jul 07 14:33:51.716329 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.716314 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2vgh2"] Jul 07 14:33:51.718310 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:51.718289 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88bef153_2452_4b8c_84f4_333d8426694d.slice/crio-928ab854da4ab35e06466440a6ec82b1596f0c8efe3b85c631ca8c153d05622d WatchSource:0}: Error finding container 928ab854da4ab35e06466440a6ec82b1596f0c8efe3b85c631ca8c153d05622d: Status 404 returned error can't find the container with id 928ab854da4ab35e06466440a6ec82b1596f0c8efe3b85c631ca8c153d05622d Jul 07 14:33:51.718678 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:51.718652 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7982337d_d41b_40b6_ab24_d925e4ac6196.slice/crio-3fca8c728c7c84b7f4de8c0a9b7a44a29f7cf6887073168b34adc435841a898a WatchSource:0}: Error finding container 3fca8c728c7c84b7f4de8c0a9b7a44a29f7cf6887073168b34adc435841a898a: Status 404 returned error can't find the container with id 3fca8c728c7c84b7f4de8c0a9b7a44a29f7cf6887073168b34adc435841a898a Jul 07 14:33:51.725359 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.725341 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" Jul 07 14:33:51.824415 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.824343 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:51.824415 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.824398 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:33:51.824614 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.824439 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:51.824614 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.824492 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:33:51.824614 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.824509 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b97d49f8-nsl7p: secret "image-registry-tls" not found Jul 07 14:33:51.824614 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.824538 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:33:51.824614 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.824544 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:33:51.824614 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.824567 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls podName:93cb5a02-4235-47be-aedc-a5d73c3d117e nodeName:}" failed. No retries permitted until 2026-07-07 14:33:52.824550795 +0000 UTC m=+38.099634204 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls") pod "image-registry-56b97d49f8-nsl7p" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e") : secret "image-registry-tls" not found Jul 07 14:33:51.824614 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.824596 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls podName:fada1bb6-9ff9-402c-b807-0a6a081e6dd0 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:52.824579126 +0000 UTC m=+38.099662538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zbvfq" (UID: "fada1bb6-9ff9-402c-b807-0a6a081e6dd0") : secret "samples-operator-tls" not found Jul 07 14:33:51.824614 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.824612 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls podName:ffc93eac-e8ab-406a-96df-22c97fec6622 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:52.824605303 +0000 UTC m=+38.099688707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4c5x2" (UID: "ffc93eac-e8ab-406a-96df-22c97fec6622") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:33:51.861327 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.861299 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8"] Jul 07 14:33:51.864198 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:33:51.864173 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27bf9b12_515c_49fd_902c_9ca037f5fa38.slice/crio-ebc6df2266c12f99abe0c21c6cef9bc15a953bae30b8b0d735c97bece1b5863a WatchSource:0}: Error finding container ebc6df2266c12f99abe0c21c6cef9bc15a953bae30b8b0d735c97bece1b5863a: Status 404 returned error can't find the container with id ebc6df2266c12f99abe0c21c6cef9bc15a953bae30b8b0d735c97bece1b5863a Jul 07 14:33:51.925135 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.925099 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.925275 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.925158 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:51.925275 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:51.925197 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:51.925275 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.925263 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:33:51.925405 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.925314 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:33:51.925405 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.925332 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:33:52.925313468 +0000 UTC m=+38.200396877 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : secret "router-metrics-certs-default" not found Jul 07 14:33:51.925405 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.925355 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls podName:adbb655f-b138-4258-909d-f74f718863a1 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:52.925342625 +0000 UTC m=+38.200426034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls") pod "dns-default-2dz7j" (UID: "adbb655f-b138-4258-909d-f74f718863a1") : secret "dns-default-metrics-tls" not found Jul 07 14:33:51.925405 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:51.925368 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:33:52.925361842 +0000 UTC m=+38.200445246 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : configmap references non-existent config key: service-ca.crt Jul 07 14:33:52.387914 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.387870 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" event={"ID":"88bef153-2452-4b8c-84f4-333d8426694d","Type":"ContainerStarted","Data":"928ab854da4ab35e06466440a6ec82b1596f0c8efe3b85c631ca8c153d05622d"} Jul 07 14:33:52.389097 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.389067 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" event={"ID":"27bf9b12-515c-49fd-902c-9ca037f5fa38","Type":"ContainerStarted","Data":"ebc6df2266c12f99abe0c21c6cef9bc15a953bae30b8b0d735c97bece1b5863a"} Jul 07 14:33:52.390278 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.390254 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2vgh2" event={"ID":"7982337d-d41b-40b6-ab24-d925e4ac6196","Type":"ContainerStarted","Data":"3fca8c728c7c84b7f4de8c0a9b7a44a29f7cf6887073168b34adc435841a898a"} Jul 07 14:33:52.391385 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.391360 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-hkxn9" event={"ID":"1fc8f4f8-d97a-4bb0-b29a-369b649e1a34","Type":"ContainerStarted","Data":"c45ca228a9515c56581a635d6bd12bb189b456de094ff16fd028ca3274d0ffbc"} Jul 07 14:33:52.392592 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.392573 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" event={"ID":"615e9275-9e38-4727-8c1f-98b3e94b86e1","Type":"ContainerStarted","Data":"1d12a8176d36613fb05f404b2a6ca15dc6e688f83df9246a6fc7359ca36acb26"} Jul 07 14:33:52.393775 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.393755 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-wdldm" event={"ID":"88545de4-3bf9-41ba-8e60-03d91dc8bd10","Type":"ContainerStarted","Data":"4b186b056a42283209549c33421e87047ff5c149de30db78791580a50264cb5e"} Jul 07 14:33:52.394863 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.394844 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" event={"ID":"dd9bcdd4-6d7b-4809-befd-7238b2ceb035","Type":"ContainerStarted","Data":"7a831e393985ca67cbc58eb33c16d510c1bdcb9cbf63d762b766c084eee8c08e"} Jul 07 14:33:52.834272 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.834177 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:52.834272 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.834269 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:33:52.834474 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.834330 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:52.835279 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.834527 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:33:52.835279 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.834595 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls podName:ffc93eac-e8ab-406a-96df-22c97fec6622 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:54.834574895 +0000 UTC m=+40.109658307 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4c5x2" (UID: "ffc93eac-e8ab-406a-96df-22c97fec6622") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:33:52.835279 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.835032 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:33:52.835279 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.835047 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b97d49f8-nsl7p: secret "image-registry-tls" not found Jul 07 14:33:52.835279 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.835091 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls podName:93cb5a02-4235-47be-aedc-a5d73c3d117e nodeName:}" failed. No retries permitted until 2026-07-07 14:33:54.835075208 +0000 UTC m=+40.110158622 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls") pod "image-registry-56b97d49f8-nsl7p" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e") : secret "image-registry-tls" not found Jul 07 14:33:52.835279 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.835154 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:33:52.835279 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.835190 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls podName:fada1bb6-9ff9-402c-b807-0a6a081e6dd0 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:54.835178647 +0000 UTC m=+40.110262065 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zbvfq" (UID: "fada1bb6-9ff9-402c-b807-0a6a081e6dd0") : secret "samples-operator-tls" not found Jul 07 14:33:52.936443 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.935515 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:52.936443 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.935594 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:52.936443 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:52.935693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:52.936443 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.935906 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:33:52.936443 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.935969 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls podName:adbb655f-b138-4258-909d-f74f718863a1 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:54.93595057 +0000 UTC m=+40.211033980 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls") pod "dns-default-2dz7j" (UID: "adbb655f-b138-4258-909d-f74f718863a1") : secret "dns-default-metrics-tls" not found Jul 07 14:33:52.936443 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.936416 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:33:54.936400776 +0000 UTC m=+40.211484187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : configmap references non-existent config key: service-ca.crt Jul 07 14:33:52.936994 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.936586 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:33:52.936994 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:52.936648 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:33:54.936633922 +0000 UTC m=+40.211717340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : secret "router-metrics-certs-default" not found Jul 07 14:33:54.854717 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:54.854678 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:54.855193 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:54.854757 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:33:54.855193 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:54.854812 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:54.855193 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.854837 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:33:54.855193 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.854859 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b97d49f8-nsl7p: secret "image-registry-tls" not found Jul 07 14:33:54.855193 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.854908 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls podName:93cb5a02-4235-47be-aedc-a5d73c3d117e nodeName:}" failed. No retries permitted until 2026-07-07 14:33:58.854892885 +0000 UTC m=+44.129976294 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls") pod "image-registry-56b97d49f8-nsl7p" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e") : secret "image-registry-tls" not found Jul 07 14:33:54.855193 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.854992 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:33:54.855193 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.855052 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls podName:ffc93eac-e8ab-406a-96df-22c97fec6622 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:58.85503479 +0000 UTC m=+44.130118208 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4c5x2" (UID: "ffc93eac-e8ab-406a-96df-22c97fec6622") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:33:54.855193 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.855088 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:33:54.855193 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.855132 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls podName:fada1bb6-9ff9-402c-b807-0a6a081e6dd0 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:58.855120099 +0000 UTC m=+44.130203520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zbvfq" (UID: "fada1bb6-9ff9-402c-b807-0a6a081e6dd0") : secret "samples-operator-tls" not found Jul 07 14:33:54.957077 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:54.956015 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:54.957077 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:54.956313 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:54.957077 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:54.956439 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:54.957077 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.956685 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:33:54.957077 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.956770 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls podName:adbb655f-b138-4258-909d-f74f718863a1 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:58.956748867 +0000 UTC m=+44.231832273 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls") pod "dns-default-2dz7j" (UID: "adbb655f-b138-4258-909d-f74f718863a1") : secret "dns-default-metrics-tls" not found Jul 07 14:33:54.957077 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.956795 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:33:58.956782991 +0000 UTC m=+44.231866399 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : configmap references non-existent config key: service-ca.crt Jul 07 14:33:54.957077 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.956883 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:33:54.957077 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:54.956944 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:33:58.956925998 +0000 UTC m=+44.232009409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : secret "router-metrics-certs-default" not found Jul 07 14:33:57.956222 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:57.956185 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:58.053471 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:58.053429 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:58.306606 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:58.306514 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:58.404931 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:58.404840 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:58.544403 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:58.544373 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:58.607745 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:58.607712 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:58.893285 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:58.893190 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:33:58.893285 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:58.893258 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:33:58.893504 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:58.893301 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:33:58.893504 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.893364 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:33:58.893504 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.893386 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b97d49f8-nsl7p: secret "image-registry-tls" not found Jul 07 14:33:58.893504 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.893411 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:33:58.893504 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.893411 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:33:58.893504 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.893454 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls podName:93cb5a02-4235-47be-aedc-a5d73c3d117e nodeName:}" failed. No retries permitted until 2026-07-07 14:34:06.893431435 +0000 UTC m=+52.168514854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls") pod "image-registry-56b97d49f8-nsl7p" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e") : secret "image-registry-tls" not found Jul 07 14:33:58.893504 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.893474 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls podName:fada1bb6-9ff9-402c-b807-0a6a081e6dd0 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:06.893464091 +0000 UTC m=+52.168547496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zbvfq" (UID: "fada1bb6-9ff9-402c-b807-0a6a081e6dd0") : secret "samples-operator-tls" not found Jul 07 14:33:58.893504 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.893489 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls podName:ffc93eac-e8ab-406a-96df-22c97fec6622 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:06.89347978 +0000 UTC m=+52.168563192 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4c5x2" (UID: "ffc93eac-e8ab-406a-96df-22c97fec6622") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:33:58.994149 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:58.994113 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:58.994507 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:58.994173 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:33:58.994507 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:33:58.994217 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:33:58.994507 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.994275 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:33:58.994507 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.994274 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:33:58.994507 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.994346 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls podName:adbb655f-b138-4258-909d-f74f718863a1 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:06.99432988 +0000 UTC m=+52.269413285 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls") pod "dns-default-2dz7j" (UID: "adbb655f-b138-4258-909d-f74f718863a1") : secret "dns-default-metrics-tls" not found Jul 07 14:33:58.994507 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.994364 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:34:06.994355162 +0000 UTC m=+52.269438566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : configmap references non-existent config key: service-ca.crt Jul 07 14:33:58.994507 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:33:58.994379 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:34:06.994370951 +0000 UTC m=+52.269454355 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : secret "router-metrics-certs-default" not found Jul 07 14:34:01.351942 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:01.351707 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:01.449601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:01.449565 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:02.241429 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.241177 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:02.353786 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.353749 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:02.429920 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.429883 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-hkxn9" event={"ID":"1fc8f4f8-d97a-4bb0-b29a-369b649e1a34","Type":"ContainerStarted","Data":"6aa5c75890441deeeae201310da1bcb32f05deb4a28ed19b3c94cb010e22fbe3"} Jul 07 14:34:02.431943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.431915 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" event={"ID":"615e9275-9e38-4727-8c1f-98b3e94b86e1","Type":"ContainerStarted","Data":"2e1653986a7de5f0f4c6a843a0c57b5465b56dcce212e7f87226746268570c98"} Jul 07 14:34:02.433709 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.433637 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-wdldm" event={"ID":"88545de4-3bf9-41ba-8e60-03d91dc8bd10","Type":"ContainerStarted","Data":"f73e3e01f7226e4eb7ba9d93acb1dacc23ee43f800c5adc52e96b065d925656c"} Jul 07 14:34:02.435297 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.435271 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" event={"ID":"dd9bcdd4-6d7b-4809-befd-7238b2ceb035","Type":"ContainerStarted","Data":"cede7d66a7df4e7434dc3ea234fad250658e7545166368ad967fa0846ebfdcdc"} Jul 07 14:34:02.436986 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.436971 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6rwzn_88bef153-2452-4b8c-84f4-333d8426694d/console-operator/0.log" Jul 07 14:34:02.437068 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.437002 2581 generic.go:356] "Generic (PLEG): container finished" podID="88bef153-2452-4b8c-84f4-333d8426694d" containerID="78ebd59eb8c60795716232521dad591546d2b798b634689876573ec47dae4175" exitCode=255 Jul 07 14:34:02.437130 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.437075 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" event={"ID":"88bef153-2452-4b8c-84f4-333d8426694d","Type":"ContainerDied","Data":"78ebd59eb8c60795716232521dad591546d2b798b634689876573ec47dae4175"} Jul 07 14:34:02.437258 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.437239 2581 scope.go:117] "RemoveContainer" containerID="78ebd59eb8c60795716232521dad591546d2b798b634689876573ec47dae4175" Jul 07 14:34:02.438734 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.438713 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" event={"ID":"27bf9b12-515c-49fd-902c-9ca037f5fa38","Type":"ContainerStarted","Data":"c55a9ed943c9cb542dc4db7d2ab83093a2e463cb24adc1f4b7ffa7339499ca55"} Jul 07 14:34:02.440861 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.440811 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2vgh2" event={"ID":"7982337d-d41b-40b6-ab24-d925e4ac6196","Type":"ContainerStarted","Data":"ce1411ae44350fcae79fc7174bf40ce4a3c23bd24c0ba9de4381d01bdcaa17ab"} Jul 07 14:34:02.443939 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.443917 2581 generic.go:356] "Generic (PLEG): container finished" podID="03c12bd0-a596-4c34-977c-3dea4a7d4fe0" containerID="49e6ee1e8769aca90d6233f7464b0d0588bceab52df28f0f81c097138533ddbc" exitCode=0 Jul 07 14:34:02.444047 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.443950 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" event={"ID":"03c12bd0-a596-4c34-977c-3dea4a7d4fe0","Type":"ContainerDied","Data":"49e6ee1e8769aca90d6233f7464b0d0588bceab52df28f0f81c097138533ddbc"} Jul 07 14:34:02.449404 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.449371 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-hkxn9" podStartSLOduration=3.276022062 podStartE2EDuration="13.449360343s" totalImagesPullingTime="10.173338281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:02.448677406 +0000 UTC m=+47.723760830" watchObservedRunningTime="2026-07-07 14:34:02.449360343 +0000 UTC m=+47.724443746" Jul 07 14:34:02.454075 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.454053 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:02.491951 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.491119 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" podStartSLOduration=3.237067732 podStartE2EDuration="13.491100693s" totalImagesPullingTime="10.254032961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:02.49107135 +0000 UTC m=+47.766154777" watchObservedRunningTime="2026-07-07 14:34:02.491100693 +0000 UTC m=+47.766184120" Jul 07 14:34:02.517503 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.517294 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" podStartSLOduration=3.330960548 podStartE2EDuration="13.517275247s" totalImagesPullingTime="10.186314699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:02.515533844 +0000 UTC m=+47.790617295" watchObservedRunningTime="2026-07-07 14:34:02.517275247 +0000 UTC m=+47.792358673" Jul 07 14:34:02.544946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.544886 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" podStartSLOduration=3.532125549 podStartE2EDuration="13.544863585s" totalImagesPullingTime="10.012738036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:02.54319722 +0000 UTC m=+47.818280647" watchObservedRunningTime="2026-07-07 14:34:02.544863585 +0000 UTC m=+47.819947012" Jul 07 14:34:02.584056 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.583487 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-wdldm" podStartSLOduration=3.541927199 podStartE2EDuration="13.583467272s" totalImagesPullingTime="10.041540073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:02.582102387 +0000 UTC m=+47.857185813" watchObservedRunningTime="2026-07-07 14:34:02.583467272 +0000 UTC m=+47.858550698" Jul 07 14:34:02.602642 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.601744 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2vgh2" podStartSLOduration=1.4440783910000001 podStartE2EDuration="11.601721748s" totalImagesPullingTime="10.157643357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:02.601131115 +0000 UTC m=+47.876214567" watchObservedRunningTime="2026-07-07 14:34:02.601721748 +0000 UTC m=+47.876805175" Jul 07 14:34:02.693580 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.693477 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:02.820115 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:02.820011 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:03.450578 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.450490 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6rwzn_88bef153-2452-4b8c-84f4-333d8426694d/console-operator/1.log" Jul 07 14:34:03.451044 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.450960 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6rwzn_88bef153-2452-4b8c-84f4-333d8426694d/console-operator/0.log" Jul 07 14:34:03.451044 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.450992 2581 generic.go:356] "Generic (PLEG): container finished" podID="88bef153-2452-4b8c-84f4-333d8426694d" containerID="e4ecb3fd2cd478fdbdd763d68c59bc3742cea6e824afce446f43a1ac3f0830fb" exitCode=255 Jul 07 14:34:03.451150 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.451096 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" event={"ID":"88bef153-2452-4b8c-84f4-333d8426694d","Type":"ContainerDied","Data":"e4ecb3fd2cd478fdbdd763d68c59bc3742cea6e824afce446f43a1ac3f0830fb"} Jul 07 14:34:03.451150 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.451145 2581 scope.go:117] "RemoveContainer" containerID="78ebd59eb8c60795716232521dad591546d2b798b634689876573ec47dae4175" Jul 07 14:34:03.451556 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.451539 2581 scope.go:117] "RemoveContainer" containerID="e4ecb3fd2cd478fdbdd763d68c59bc3742cea6e824afce446f43a1ac3f0830fb" Jul 07 14:34:03.452620 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:03.451800 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-6rwzn_openshift-console-operator(88bef153-2452-4b8c-84f4-333d8426694d)\"" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" podUID="88bef153-2452-4b8c-84f4-333d8426694d" Jul 07 14:34:03.456267 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.456240 2581 generic.go:356] "Generic (PLEG): container finished" podID="03c12bd0-a596-4c34-977c-3dea4a7d4fe0" containerID="e4de7b1fe898ab6e3bb0ffb5eb3c06624016479c2c15e409c9d82f9bb992c1aa" exitCode=0 Jul 07 14:34:03.456908 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.456885 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" event={"ID":"03c12bd0-a596-4c34-977c-3dea4a7d4fe0","Type":"ContainerDied","Data":"e4de7b1fe898ab6e3bb0ffb5eb3c06624016479c2c15e409c9d82f9bb992c1aa"} Jul 07 14:34:03.513478 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.513449 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:03.594475 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.594441 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gznjd"] Jul 07 14:34:03.621867 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.621790 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gznjd"] Jul 07 14:34:03.622018 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.621955 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.626042 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.626018 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 14:34:03.626183 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.626133 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 14:34:03.626183 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.626156 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nl4cp\"" Jul 07 14:34:03.663271 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.663160 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:03.741420 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.741345 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:03.744652 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.744635 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ad450ca-750b-4c5a-a5c0-bc66602536f7-data-volume\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.744738 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.744725 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.744777 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.744746 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.744777 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.744774 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k4ff\" (UniqueName: \"kubernetes.io/projected/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-api-access-8k4ff\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.744870 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.744797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8ad450ca-750b-4c5a-a5c0-bc66602536f7-crio-socket\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.846171 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.846133 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.846171 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.846171 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.846436 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.846203 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k4ff\" (UniqueName: \"kubernetes.io/projected/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-api-access-8k4ff\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.846436 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:03.846273 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 14:34:03.846436 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:03.846303 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:34:03.846436 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.846331 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8ad450ca-750b-4c5a-a5c0-bc66602536f7-crio-socket\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.846436 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:03.846346 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm podName:8ad450ca-750b-4c5a-a5c0-bc66602536f7 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:04.346326035 +0000 UTC m=+49.621409450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gznjd" (UID: "8ad450ca-750b-4c5a-a5c0-bc66602536f7") : configmap "kube-rbac-proxy" not found Jul 07 14:34:03.846436 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:03.846414 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls podName:8ad450ca-750b-4c5a-a5c0-bc66602536f7 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:04.346391817 +0000 UTC m=+49.621475224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gznjd" (UID: "8ad450ca-750b-4c5a-a5c0-bc66602536f7") : secret "insights-runtime-extractor-tls" not found Jul 07 14:34:03.846625 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.846493 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ad450ca-750b-4c5a-a5c0-bc66602536f7-data-volume\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.846625 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.846549 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8ad450ca-750b-4c5a-a5c0-bc66602536f7-crio-socket\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.846804 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.846788 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ad450ca-750b-4c5a-a5c0-bc66602536f7-data-volume\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.857006 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.856983 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k4ff\" (UniqueName: \"kubernetes.io/projected/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-api-access-8k4ff\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:03.872510 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.872480 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:03.975809 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:03.975775 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:04.013292 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.013219 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67"] Jul 07 14:34:04.033605 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.033574 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67"] Jul 07 14:34:04.033740 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.033721 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67" Jul 07 14:34:04.036455 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.036432 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 07 14:34:04.036455 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.036445 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 07 14:34:04.036625 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.036474 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gkrz2\"" Jul 07 14:34:04.087499 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.087470 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:04.150043 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.150011 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44sn6\" (UniqueName: \"kubernetes.io/projected/c8cf2483-6838-403c-bb44-abc6adbad239-kube-api-access-44sn6\") pod \"migrator-d57d58cc9-72k67\" (UID: \"c8cf2483-6838-403c-bb44-abc6adbad239\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67" Jul 07 14:34:04.183209 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.183178 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:04.251305 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.251262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44sn6\" (UniqueName: \"kubernetes.io/projected/c8cf2483-6838-403c-bb44-abc6adbad239-kube-api-access-44sn6\") pod \"migrator-d57d58cc9-72k67\" (UID: \"c8cf2483-6838-403c-bb44-abc6adbad239\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67" Jul 07 14:34:04.260023 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.259984 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44sn6\" (UniqueName: \"kubernetes.io/projected/c8cf2483-6838-403c-bb44-abc6adbad239-kube-api-access-44sn6\") pod \"migrator-d57d58cc9-72k67\" (UID: \"c8cf2483-6838-403c-bb44-abc6adbad239\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67" Jul 07 14:34:04.343697 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.343671 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67" Jul 07 14:34:04.352170 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.352146 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:04.352287 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.352180 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:04.352331 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:04.352299 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 14:34:04.352375 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:04.352363 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm podName:8ad450ca-750b-4c5a-a5c0-bc66602536f7 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:05.352347782 +0000 UTC m=+50.627431187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gznjd" (UID: "8ad450ca-750b-4c5a-a5c0-bc66602536f7") : configmap "kube-rbac-proxy" not found Jul 07 14:34:04.352417 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:04.352303 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:34:04.352446 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:04.352417 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls podName:8ad450ca-750b-4c5a-a5c0-bc66602536f7 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:05.352406123 +0000 UTC m=+50.627489533 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gznjd" (UID: "8ad450ca-750b-4c5a-a5c0-bc66602536f7") : secret "insights-runtime-extractor-tls" not found Jul 07 14:34:04.462480 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.462441 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6rwzn_88bef153-2452-4b8c-84f4-333d8426694d/console-operator/1.log" Jul 07 14:34:04.462909 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.462796 2581 scope.go:117] "RemoveContainer" containerID="e4ecb3fd2cd478fdbdd763d68c59bc3742cea6e824afce446f43a1ac3f0830fb" Jul 07 14:34:04.463233 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:04.463008 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-6rwzn_openshift-console-operator(88bef153-2452-4b8c-84f4-333d8426694d)\"" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" podUID="88bef153-2452-4b8c-84f4-333d8426694d" Jul 07 14:34:04.467602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.467575 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" event={"ID":"03c12bd0-a596-4c34-977c-3dea4a7d4fe0","Type":"ContainerStarted","Data":"f67d555acce8cd03f4a08ff216fd6944a58868499b8600d263e0ab7015ed1768"} Jul 07 14:34:04.512653 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.512624 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67"] Jul 07 14:34:04.516156 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:34:04.516130 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8cf2483_6838_403c_bb44_abc6adbad239.slice/crio-4abe097e7be01b4e0c0ba4b813f43451a920fa68a60e84011eac7c447e172138 WatchSource:0}: Error finding container 4abe097e7be01b4e0c0ba4b813f43451a920fa68a60e84011eac7c447e172138: Status 404 returned error can't find the container with id 4abe097e7be01b4e0c0ba4b813f43451a920fa68a60e84011eac7c447e172138 Jul 07 14:34:04.525719 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.525696 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:04.568010 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.567977 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-92t44_9ecd6676-11b6-49ad-b15f-e08fbc43c0b6/dns-node-resolver/0.log" Jul 07 14:34:04.662361 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.662286 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:04.763391 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:04.763357 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:05.295183 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:05.295152 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:05.362930 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:05.362896 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:05.362930 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:05.362933 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:05.363125 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:05.363040 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:34:05.363125 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:05.363077 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 14:34:05.363125 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:05.363120 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls podName:8ad450ca-750b-4c5a-a5c0-bc66602536f7 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:07.363101584 +0000 UTC m=+52.638184990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gznjd" (UID: "8ad450ca-750b-4c5a-a5c0-bc66602536f7") : secret "insights-runtime-extractor-tls" not found Jul 07 14:34:05.363226 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:05.363136 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm podName:8ad450ca-750b-4c5a-a5c0-bc66602536f7 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:07.363129732 +0000 UTC m=+52.638213137 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gznjd" (UID: "8ad450ca-750b-4c5a-a5c0-bc66602536f7") : configmap "kube-rbac-proxy" not found Jul 07 14:34:05.369150 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:05.369129 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6hfkd_17ef7399-7fb7-4446-b7b5-c85cc6d7e43e/node-ca/0.log" Jul 07 14:34:05.472396 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:05.472359 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67" event={"ID":"c8cf2483-6838-403c-bb44-abc6adbad239","Type":"ContainerStarted","Data":"4abe097e7be01b4e0c0ba4b813f43451a920fa68a60e84011eac7c447e172138"} Jul 07 14:34:06.106702 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.106641 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xnhfr" podStartSLOduration=17.600727203 podStartE2EDuration="50.106622077s" totalImagesPullingTime="32.505894874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:04.523335907 +0000 UTC m=+49.798419333" watchObservedRunningTime="2026-07-07 14:34:06.106622077 +0000 UTC m=+51.381705480" Jul 07 14:34:06.107489 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.107461 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-bxc9h"] Jul 07 14:34:06.122786 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.122760 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-bxc9h"] Jul 07 14:34:06.122951 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.122904 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-bxc9h" Jul 07 14:34:06.126222 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.126200 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 07 14:34:06.126341 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.126229 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 07 14:34:06.127284 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.127251 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qcwvr\"" Jul 07 14:34:06.127284 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.127281 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 07 14:34:06.127436 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.127366 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 07 14:34:06.152511 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.152480 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:06.168402 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.168375 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2vgh2_7982337d-d41b-40b6-ab24-d925e4ac6196/serve-healthcheck-canary/0.log" Jul 07 14:34:06.222571 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.222538 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:06.272912 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.272874 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5830840f-adf3-4720-800c-6e371fd95527-signing-cabundle\") pod \"service-ca-758f599c5-bxc9h\" (UID: \"5830840f-adf3-4720-800c-6e371fd95527\") " pod="openshift-service-ca/service-ca-758f599c5-bxc9h" Jul 07 14:34:06.273072 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.273002 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5830840f-adf3-4720-800c-6e371fd95527-signing-key\") pod \"service-ca-758f599c5-bxc9h\" (UID: \"5830840f-adf3-4720-800c-6e371fd95527\") " pod="openshift-service-ca/service-ca-758f599c5-bxc9h" Jul 07 14:34:06.273148 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.273079 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gpwr\" (UniqueName: \"kubernetes.io/projected/5830840f-adf3-4720-800c-6e371fd95527-kube-api-access-9gpwr\") pod \"service-ca-758f599c5-bxc9h\" (UID: \"5830840f-adf3-4720-800c-6e371fd95527\") " pod="openshift-service-ca/service-ca-758f599c5-bxc9h" Jul 07 14:34:06.374518 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.374435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5830840f-adf3-4720-800c-6e371fd95527-signing-key\") pod \"service-ca-758f599c5-bxc9h\" (UID: \"5830840f-adf3-4720-800c-6e371fd95527\") " pod="openshift-service-ca/service-ca-758f599c5-bxc9h" Jul 07 14:34:06.374685 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.374523 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gpwr\" (UniqueName: \"kubernetes.io/projected/5830840f-adf3-4720-800c-6e371fd95527-kube-api-access-9gpwr\") pod \"service-ca-758f599c5-bxc9h\" (UID: \"5830840f-adf3-4720-800c-6e371fd95527\") " pod="openshift-service-ca/service-ca-758f599c5-bxc9h" Jul 07 14:34:06.374685 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.374616 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5830840f-adf3-4720-800c-6e371fd95527-signing-cabundle\") pod \"service-ca-758f599c5-bxc9h\" (UID: \"5830840f-adf3-4720-800c-6e371fd95527\") " pod="openshift-service-ca/service-ca-758f599c5-bxc9h" Jul 07 14:34:06.375269 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.375249 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5830840f-adf3-4720-800c-6e371fd95527-signing-cabundle\") pod \"service-ca-758f599c5-bxc9h\" (UID: \"5830840f-adf3-4720-800c-6e371fd95527\") " pod="openshift-service-ca/service-ca-758f599c5-bxc9h" Jul 07 14:34:06.376764 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.376747 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5830840f-adf3-4720-800c-6e371fd95527-signing-key\") pod \"service-ca-758f599c5-bxc9h\" (UID: \"5830840f-adf3-4720-800c-6e371fd95527\") " pod="openshift-service-ca/service-ca-758f599c5-bxc9h" Jul 07 14:34:06.385105 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.385084 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gpwr\" (UniqueName: \"kubernetes.io/projected/5830840f-adf3-4720-800c-6e371fd95527-kube-api-access-9gpwr\") pod \"service-ca-758f599c5-bxc9h\" (UID: \"5830840f-adf3-4720-800c-6e371fd95527\") " pod="openshift-service-ca/service-ca-758f599c5-bxc9h" Jul 07 14:34:06.434217 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.434190 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-bxc9h" Jul 07 14:34:06.583293 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.583267 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-bxc9h"] Jul 07 14:34:06.586156 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:34:06.586133 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5830840f_adf3_4720_800c_6e371fd95527.slice/crio-153d390e6bd2ac6a32c0814303dad791d27ad6edad6483f6f882c529a756cf5a WatchSource:0}: Error finding container 153d390e6bd2ac6a32c0814303dad791d27ad6edad6483f6f882c529a756cf5a: Status 404 returned error can't find the container with id 153d390e6bd2ac6a32c0814303dad791d27ad6edad6483f6f882c529a756cf5a Jul 07 14:34:06.979654 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.979572 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:34:06.979801 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.979663 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:34:06.979801 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:06.979720 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:34:06.979801 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:06.979740 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56b97d49f8-nsl7p: secret "image-registry-tls" not found Jul 07 14:34:06.979801 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:06.979789 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls podName:93cb5a02-4235-47be-aedc-a5d73c3d117e nodeName:}" failed. No retries permitted until 2026-07-07 14:34:22.979773218 +0000 UTC m=+68.254856625 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls") pod "image-registry-56b97d49f8-nsl7p" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e") : secret "image-registry-tls" not found Jul 07 14:34:06.979960 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:06.979810 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:34:06.979960 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:06.979890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:34:06.979960 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:06.979898 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls podName:fada1bb6-9ff9-402c-b807-0a6a081e6dd0 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:22.979879797 +0000 UTC m=+68.254963221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zbvfq" (UID: "fada1bb6-9ff9-402c-b807-0a6a081e6dd0") : secret "samples-operator-tls" not found Jul 07 14:34:06.980060 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:06.979963 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:34:06.980060 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:06.979997 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls podName:ffc93eac-e8ab-406a-96df-22c97fec6622 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:22.979982655 +0000 UTC m=+68.255066059 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4c5x2" (UID: "ffc93eac-e8ab-406a-96df-22c97fec6622") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:34:07.080928 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.080893 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:34:07.081093 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.080992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:34:07.081093 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.081033 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:34:07.081093 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:07.081061 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:34:23.081043062 +0000 UTC m=+68.356126466 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : configmap references non-existent config key: service-ca.crt Jul 07 14:34:07.081232 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:07.081123 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:34:07.081232 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:07.081143 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:34:07.081232 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:07.081170 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls podName:adbb655f-b138-4258-909d-f74f718863a1 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:23.081155502 +0000 UTC m=+68.356238908 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls") pod "dns-default-2dz7j" (UID: "adbb655f-b138-4258-909d-f74f718863a1") : secret "dns-default-metrics-tls" not found Jul 07 14:34:07.081232 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:07.081196 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs podName:2485c568-9c6b-4bfe-8a8e-9437042d1f9f nodeName:}" failed. No retries permitted until 2026-07-07 14:34:23.081181295 +0000 UTC m=+68.356264713 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs") pod "router-default-67bf4497f9-krpmz" (UID: "2485c568-9c6b-4bfe-8a8e-9437042d1f9f") : secret "router-metrics-certs-default" not found Jul 07 14:34:07.149435 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.149397 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:07.384143 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.384107 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:07.384290 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.384152 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:07.384290 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:07.384272 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 14:34:07.384374 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:07.384284 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:34:07.384374 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:07.384338 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm podName:8ad450ca-750b-4c5a-a5c0-bc66602536f7 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:11.384320725 +0000 UTC m=+56.659404129 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gznjd" (UID: "8ad450ca-750b-4c5a-a5c0-bc66602536f7") : configmap "kube-rbac-proxy" not found Jul 07 14:34:07.384374 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:07.384355 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls podName:8ad450ca-750b-4c5a-a5c0-bc66602536f7 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:11.384346499 +0000 UTC m=+56.659429909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gznjd" (UID: "8ad450ca-750b-4c5a-a5c0-bc66602536f7") : secret "insights-runtime-extractor-tls" not found Jul 07 14:34:07.481195 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.481153 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67" event={"ID":"c8cf2483-6838-403c-bb44-abc6adbad239","Type":"ContainerStarted","Data":"29caeb3e000efeef18941f714993c88047414b4c15a727bb75cf23236c7a7fdf"} Jul 07 14:34:07.481374 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.481215 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67" event={"ID":"c8cf2483-6838-403c-bb44-abc6adbad239","Type":"ContainerStarted","Data":"7338c06276e28eb0e26b68d32049fcc1d40d60b2f0c85183fb73b3313e8df3be"} Jul 07 14:34:07.482697 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.482677 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-bxc9h" event={"ID":"5830840f-adf3-4720-800c-6e371fd95527","Type":"ContainerStarted","Data":"50ecc4ee21bde1806b05e0da395e3fdba2f4b95ddba8dd586c7240f03595e42a"} Jul 07 14:34:07.482791 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.482711 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-bxc9h" event={"ID":"5830840f-adf3-4720-800c-6e371fd95527","Type":"ContainerStarted","Data":"153d390e6bd2ac6a32c0814303dad791d27ad6edad6483f6f882c529a756cf5a"} Jul 07 14:34:07.511134 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.511083 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-72k67" podStartSLOduration=2.572102477 podStartE2EDuration="4.511069473s" totalImagesPullingTime="1.938966996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:07.510245366 +0000 UTC m=+52.785328792" watchObservedRunningTime="2026-07-07 14:34:07.511069473 +0000 UTC m=+52.786152900" Jul 07 14:34:07.531357 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:07.531313 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-bxc9h" podStartSLOduration=1.531299915 podStartE2EDuration="1.531299915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:07.531279173 +0000 UTC m=+52.806362598" watchObservedRunningTime="2026-07-07 14:34:07.531299915 +0000 UTC m=+52.806383341" Jul 07 14:34:08.444856 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:08.444807 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:08.565235 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:08.565201 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:08.665257 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:08.665220 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:08.764289 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:08.764183 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:08.879445 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:08.879410 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:09.180345 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:09.180314 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:10.141478 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:10.141448 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:10.549373 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:10.549288 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:10.632419 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:10.632380 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:10.741422 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:10.741387 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:10.810994 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:10.810904 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:10.940292 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:10.940258 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:11.335347 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:11.335313 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:11.421393 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:11.421358 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:11.421393 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:11.421394 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:11.421606 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:11.421541 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:34:11.421606 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:11.421550 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 14:34:11.421678 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:11.421618 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls podName:8ad450ca-750b-4c5a-a5c0-bc66602536f7 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:19.42159752 +0000 UTC m=+64.696680939 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gznjd" (UID: "8ad450ca-750b-4c5a-a5c0-bc66602536f7") : secret "insights-runtime-extractor-tls" not found Jul 07 14:34:11.421678 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:11.421640 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm podName:8ad450ca-750b-4c5a-a5c0-bc66602536f7 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:19.421627963 +0000 UTC m=+64.696711367 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gznjd" (UID: "8ad450ca-750b-4c5a-a5c0-bc66602536f7") : configmap "kube-rbac-proxy" not found Jul 07 14:34:11.472359 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:11.472313 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:34:11.472494 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:11.472389 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:34:11.472758 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:11.472745 2581 scope.go:117] "RemoveContainer" containerID="e4ecb3fd2cd478fdbdd763d68c59bc3742cea6e824afce446f43a1ac3f0830fb" Jul 07 14:34:11.472935 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:34:11.472917 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-6rwzn_openshift-console-operator(88bef153-2452-4b8c-84f4-333d8426694d)\"" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" podUID="88bef153-2452-4b8c-84f4-333d8426694d" Jul 07 14:34:11.735387 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:11.735303 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:11.997531 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:11.997450 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:12.347589 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:12.347561 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:12.534961 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:12.534929 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:12.751763 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:12.751689 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:12.855049 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:12.855018 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:13.156470 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:13.156431 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:14.099993 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:14.099961 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:14.168442 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:14.168413 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:14.540303 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:14.540224 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:14.735400 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:14.735370 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:15.137207 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:15.137179 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:15.252454 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:15.252421 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:15.350059 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:15.350025 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:16.378304 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:16.378279 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4k7w" Jul 07 14:34:16.853555 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:16.853517 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:16.954885 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:16.954847 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:17.132059 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:17.131941 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:17.540329 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:17.540224 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:17.946019 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:17.945986 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:18.340732 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:18.340685 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:19.355880 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:19.355847 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:19.424113 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:19.424081 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:19.492862 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:19.492813 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:19.492862 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:19.492863 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:19.493356 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:19.493339 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ad450ca-750b-4c5a-a5c0-bc66602536f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:19.495130 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:19.495111 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ad450ca-750b-4c5a-a5c0-bc66602536f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gznjd\" (UID: \"8ad450ca-750b-4c5a-a5c0-bc66602536f7\") " pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:19.536697 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:19.536670 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nl4cp\"" Jul 07 14:34:19.544949 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:19.544919 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gznjd" Jul 07 14:34:19.689342 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:19.689310 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gznjd"] Jul 07 14:34:19.692329 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:34:19.692302 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad450ca_750b_4c5a_a5c0_bc66602536f7.slice/crio-411505a04f9608749af216e54ffa6ab113a426d35763960268322d67a77abc7f WatchSource:0}: Error finding container 411505a04f9608749af216e54ffa6ab113a426d35763960268322d67a77abc7f: Status 404 returned error can't find the container with id 411505a04f9608749af216e54ffa6ab113a426d35763960268322d67a77abc7f Jul 07 14:34:20.528607 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:20.528521 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gznjd" event={"ID":"8ad450ca-750b-4c5a-a5c0-bc66602536f7","Type":"ContainerStarted","Data":"411505a04f9608749af216e54ffa6ab113a426d35763960268322d67a77abc7f"} Jul 07 14:34:20.906598 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:20.906553 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:34:20.909649 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:20.909624 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 14:34:20.919371 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:20.919346 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f5b7926-67d9-4e32-bc4e-0c14a42118ea-metrics-certs\") pod \"network-metrics-daemon-mzxzk\" (UID: \"0f5b7926-67d9-4e32-bc4e-0c14a42118ea\") " pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:34:20.962439 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:20.962412 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bf67t\"" Jul 07 14:34:20.970467 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:20.970427 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mzxzk" Jul 07 14:34:21.007317 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.007288 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:34:21.010105 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.010082 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 14:34:21.021101 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.021077 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91b21080-c287-47e7-9829-601c1be484f2-original-pull-secret\") pod \"global-pull-secret-syncer-xb5ll\" (UID: \"91b21080-c287-47e7-9829-601c1be484f2\") " pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:34:21.108101 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.108066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pn6\" (UniqueName: \"kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6\") pod \"network-check-target-jrxsr\" (UID: \"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4\") " pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:34:21.111069 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.111045 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8pn6\" (UniqueName: \"kubernetes.io/projected/4e7b899b-9bd2-4fb6-907c-69e88b59c1b4-kube-api-access-z8pn6\") pod \"network-check-target-jrxsr\" (UID: \"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4\") " pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:34:21.133417 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.133388 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mzxzk"] Jul 07 14:34:21.136447 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:34:21.136423 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f5b7926_67d9_4e32_bc4e_0c14a42118ea.slice/crio-2b35e161b247ddc8f218235188034e8dbb99be7ecd168f7adb153a6e003c2633 WatchSource:0}: Error finding container 2b35e161b247ddc8f218235188034e8dbb99be7ecd168f7adb153a6e003c2633: Status 404 returned error can't find the container with id 2b35e161b247ddc8f218235188034e8dbb99be7ecd168f7adb153a6e003c2633 Jul 07 14:34:21.251838 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.251743 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fmsjq\"" Jul 07 14:34:21.254975 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.254954 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xb5ll" Jul 07 14:34:21.259961 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.259941 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:34:21.430022 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.429991 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xb5ll"] Jul 07 14:34:21.447299 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.447184 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jrxsr"] Jul 07 14:34:21.534103 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:21.534026 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mzxzk" event={"ID":"0f5b7926-67d9-4e32-bc4e-0c14a42118ea","Type":"ContainerStarted","Data":"2b35e161b247ddc8f218235188034e8dbb99be7ecd168f7adb153a6e003c2633"} Jul 07 14:34:22.642263 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:34:22.642228 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e7b899b_9bd2_4fb6_907c_69e88b59c1b4.slice/crio-e3f76c5f8a374ee15243a5e286b2ac3a336f19358c899999cada8bc92e1c27ef WatchSource:0}: Error finding container e3f76c5f8a374ee15243a5e286b2ac3a336f19358c899999cada8bc92e1c27ef: Status 404 returned error can't find the container with id e3f76c5f8a374ee15243a5e286b2ac3a336f19358c899999cada8bc92e1c27ef Jul 07 14:34:22.643261 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:34:22.642951 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b21080_c287_47e7_9829_601c1be484f2.slice/crio-70e674a4986fde1daa22872aae462b02f90fdf00137300dfe56116cc0c4a48db WatchSource:0}: Error finding container 70e674a4986fde1daa22872aae462b02f90fdf00137300dfe56116cc0c4a48db: Status 404 returned error can't find the container with id 70e674a4986fde1daa22872aae462b02f90fdf00137300dfe56116cc0c4a48db Jul 07 14:34:23.025917 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.025803 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:34:23.025917 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.025911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:34:23.026154 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.025971 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:34:23.028883 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.028858 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffc93eac-e8ab-406a-96df-22c97fec6622-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4c5x2\" (UID: \"ffc93eac-e8ab-406a-96df-22c97fec6622\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:34:23.028998 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.028963 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls\") pod \"image-registry-56b97d49f8-nsl7p\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:34:23.029374 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.029332 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fada1bb6-9ff9-402c-b807-0a6a081e6dd0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zbvfq\" (UID: \"fada1bb6-9ff9-402c-b807-0a6a081e6dd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:34:23.127061 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.127021 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:34:23.127243 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.127087 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:34:23.127243 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.127130 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:34:23.127756 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.127731 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-service-ca-bundle\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:34:23.129552 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.129517 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/adbb655f-b138-4258-909d-f74f718863a1-metrics-tls\") pod \"dns-default-2dz7j\" (UID: \"adbb655f-b138-4258-909d-f74f718863a1\") " pod="openshift-dns/dns-default-2dz7j" Jul 07 14:34:23.129670 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.129649 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2485c568-9c6b-4bfe-8a8e-9437042d1f9f-metrics-certs\") pod \"router-default-67bf4497f9-krpmz\" (UID: \"2485c568-9c6b-4bfe-8a8e-9437042d1f9f\") " pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:34:23.179148 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.179115 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qgdmq\"" Jul 07 14:34:23.186717 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.186687 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:34:23.200977 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.200955 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ncfp7\"" Jul 07 14:34:23.208266 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.208238 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" Jul 07 14:34:23.213115 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.213095 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-45kw9\"" Jul 07 14:34:23.221562 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.221537 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" Jul 07 14:34:23.234994 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.234973 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-75lw4\"" Jul 07 14:34:23.242643 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.242622 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:34:23.300053 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.300029 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bxlxt\"" Jul 07 14:34:23.308146 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.308116 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2dz7j" Jul 07 14:34:23.542608 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.542514 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xb5ll" event={"ID":"91b21080-c287-47e7-9829-601c1be484f2","Type":"ContainerStarted","Data":"70e674a4986fde1daa22872aae462b02f90fdf00137300dfe56116cc0c4a48db"} Jul 07 14:34:23.543739 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.543711 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jrxsr" event={"ID":"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4","Type":"ContainerStarted","Data":"e3f76c5f8a374ee15243a5e286b2ac3a336f19358c899999cada8bc92e1c27ef"} Jul 07 14:34:23.878721 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.878669 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56b97d49f8-nsl7p"] Jul 07 14:34:23.903319 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.903287 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67bf4497f9-krpmz"] Jul 07 14:34:23.916525 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.916500 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq"] Jul 07 14:34:23.940653 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:23.940620 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2dz7j"] Jul 07 14:34:24.044137 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.044022 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2"] Jul 07 14:34:24.051350 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:34:24.051306 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93cb5a02_4235_47be_aedc_a5d73c3d117e.slice/crio-c8aee38519be0b06df91d30f31865695961f272e637ae0f2adf1fbb4639207fa WatchSource:0}: Error finding container c8aee38519be0b06df91d30f31865695961f272e637ae0f2adf1fbb4639207fa: Status 404 returned error can't find the container with id c8aee38519be0b06df91d30f31865695961f272e637ae0f2adf1fbb4639207fa Jul 07 14:34:24.052374 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:34:24.052325 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadbb655f_b138_4258_909d_f74f718863a1.slice/crio-acc0814a8863905bdee5d84ff8d481824c1f43c80ca054ab064b767948c2e8dd WatchSource:0}: Error finding container acc0814a8863905bdee5d84ff8d481824c1f43c80ca054ab064b767948c2e8dd: Status 404 returned error can't find the container with id acc0814a8863905bdee5d84ff8d481824c1f43c80ca054ab064b767948c2e8dd Jul 07 14:34:24.053692 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:34:24.053621 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2485c568_9c6b_4bfe_8a8e_9437042d1f9f.slice/crio-2241ea39e8bd038d48686a9f4c4c225e238abc3984e3ec1279dd7a251bb5b740 WatchSource:0}: Error finding container 2241ea39e8bd038d48686a9f4c4c225e238abc3984e3ec1279dd7a251bb5b740: Status 404 returned error can't find the container with id 2241ea39e8bd038d48686a9f4c4c225e238abc3984e3ec1279dd7a251bb5b740 Jul 07 14:34:24.236246 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.236225 2581 scope.go:117] "RemoveContainer" containerID="e4ecb3fd2cd478fdbdd763d68c59bc3742cea6e824afce446f43a1ac3f0830fb" Jul 07 14:34:24.552879 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.552761 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gznjd" event={"ID":"8ad450ca-750b-4c5a-a5c0-bc66602536f7","Type":"ContainerStarted","Data":"5ba33f3163b49be8434bcffd1a4fa8dee0bc78da41956ac67ece9c36a47b6420"} Jul 07 14:34:24.555020 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.554990 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67bf4497f9-krpmz" event={"ID":"2485c568-9c6b-4bfe-8a8e-9437042d1f9f","Type":"ContainerStarted","Data":"ab98ca8fac4fe197cf0b3b03ab144a278b4add27f4841f533a01b826e61c6b8f"} Jul 07 14:34:24.555151 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.555039 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67bf4497f9-krpmz" event={"ID":"2485c568-9c6b-4bfe-8a8e-9437042d1f9f","Type":"ContainerStarted","Data":"2241ea39e8bd038d48686a9f4c4c225e238abc3984e3ec1279dd7a251bb5b740"} Jul 07 14:34:24.556894 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.556843 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jrxsr" event={"ID":"4e7b899b-9bd2-4fb6-907c-69e88b59c1b4","Type":"ContainerStarted","Data":"1e9710e5849b1534486571c98f750ad3842dff80b84647f9d6c753f93f22defe"} Jul 07 14:34:24.557000 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.556908 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:34:24.559178 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.559099 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mzxzk" event={"ID":"0f5b7926-67d9-4e32-bc4e-0c14a42118ea","Type":"ContainerStarted","Data":"ec6f450e5746921bb395eb5f142046ef0c51d645461fc617988cdbcbaa93bd7d"} Jul 07 14:34:24.559178 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.559143 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mzxzk" event={"ID":"0f5b7926-67d9-4e32-bc4e-0c14a42118ea","Type":"ContainerStarted","Data":"38c2dff7996d805594344e9cbeedcc8bd1fc95bab7296127b81e07c0894141ee"} Jul 07 14:34:24.560747 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.560699 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2dz7j" event={"ID":"adbb655f-b138-4258-909d-f74f718863a1","Type":"ContainerStarted","Data":"acc0814a8863905bdee5d84ff8d481824c1f43c80ca054ab064b767948c2e8dd"} Jul 07 14:34:24.562075 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.562037 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" event={"ID":"fada1bb6-9ff9-402c-b807-0a6a081e6dd0","Type":"ContainerStarted","Data":"2d91b3e5d8a949574c9c3150eab01cb6271315c35a1bdeb4c407943a96da0b57"} Jul 07 14:34:24.563455 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.563432 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" event={"ID":"ffc93eac-e8ab-406a-96df-22c97fec6622","Type":"ContainerStarted","Data":"f20f4feba98cc0ad1133ce1b0b72acaa1c487c28bf83595a1ce9e3df0c674831"} Jul 07 14:34:24.565567 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.565545 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6rwzn_88bef153-2452-4b8c-84f4-333d8426694d/console-operator/1.log" Jul 07 14:34:24.565658 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.565628 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" event={"ID":"88bef153-2452-4b8c-84f4-333d8426694d","Type":"ContainerStarted","Data":"b7b737db469de0aabc0b3398d485d03c7e1257eca70a50fa585098fba29010f3"} Jul 07 14:34:24.566410 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.565970 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:34:24.567808 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.567782 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" event={"ID":"93cb5a02-4235-47be-aedc-a5d73c3d117e","Type":"ContainerStarted","Data":"58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb"} Jul 07 14:34:24.567923 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.567838 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" event={"ID":"93cb5a02-4235-47be-aedc-a5d73c3d117e","Type":"ContainerStarted","Data":"c8aee38519be0b06df91d30f31865695961f272e637ae0f2adf1fbb4639207fa"} Jul 07 14:34:24.567981 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.567943 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:34:24.579513 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.579037 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-67bf4497f9-krpmz" podStartSLOduration=35.579018982 podStartE2EDuration="35.579018982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:24.57885475 +0000 UTC m=+69.853938203" watchObservedRunningTime="2026-07-07 14:34:24.579018982 +0000 UTC m=+69.854102411" Jul 07 14:34:24.654529 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.654160 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" podStartSLOduration=25.495836413 podStartE2EDuration="35.654138626s" totalImagesPullingTime="10.158302213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:24.610029321 +0000 UTC m=+69.885112748" watchObservedRunningTime="2026-07-07 14:34:24.654138626 +0000 UTC m=+69.929222051" Jul 07 14:34:24.654709 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.654590 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mzxzk" podStartSLOduration=65.681337974 podStartE2EDuration="1m8.654576819s" totalImagesPullingTime="2.973238845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:24.640898846 +0000 UTC m=+69.915982298" watchObservedRunningTime="2026-07-07 14:34:24.654576819 +0000 UTC m=+69.929660249" Jul 07 14:34:24.698528 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.696929 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jrxsr" podStartSLOduration=68.696910348 podStartE2EDuration="1m8.696910348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:24.696667184 +0000 UTC m=+69.971750622" watchObservedRunningTime="2026-07-07 14:34:24.696910348 +0000 UTC m=+69.971993777" Jul 07 14:34:24.698528 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.697540 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" podStartSLOduration=68.697529353 podStartE2EDuration="1m8.697529353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:24.67815871 +0000 UTC m=+69.953242157" watchObservedRunningTime="2026-07-07 14:34:24.697529353 +0000 UTC m=+69.972612780" Jul 07 14:34:24.896622 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:24.896593 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-6rwzn" Jul 07 14:34:25.091261 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.091224 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:25.196442 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.196405 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:25.243258 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.243231 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:34:25.246720 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.246697 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:34:25.276084 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.276047 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:25.381845 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.381672 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:25.559063 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.559030 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:25.582846 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.582241 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gznjd" event={"ID":"8ad450ca-750b-4c5a-a5c0-bc66602536f7","Type":"ContainerStarted","Data":"185bb3218c2cb62a41219a742de72b89ebe0f6491f631549200d0439fb5e0caa"} Jul 07 14:34:25.584829 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.584310 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:34:25.586209 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.586030 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-67bf4497f9-krpmz" Jul 07 14:34:25.636066 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.635934 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:25.962484 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:25.962405 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:26.065311 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.065219 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:26.166903 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.166792 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:26.265387 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.265310 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:26.575672 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.575553 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj"] Jul 07 14:34:26.580445 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.579880 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj" Jul 07 14:34:26.585173 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.584733 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-svdxf\"" Jul 07 14:34:26.585173 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.584851 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 14:34:26.585173 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.585037 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 14:34:26.589656 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.589066 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj"] Jul 07 14:34:26.616786 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.616754 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:26.662922 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.662878 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qd7pj\" (UID: \"9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj" Jul 07 14:34:26.663128 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.662942 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qd7pj\" (UID: \"9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj" Jul 07 14:34:26.689143 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.689109 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-vngwn"] Jul 07 14:34:26.692988 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.692962 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-vngwn" Jul 07 14:34:26.697573 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.697529 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 14:34:26.698153 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.698129 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-xfppx\"" Jul 07 14:34:26.698364 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.698333 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 14:34:26.706881 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.706807 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-vngwn"] Jul 07 14:34:26.740763 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.740727 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:26.764344 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.764286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qd7pj\" (UID: \"9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj" Jul 07 14:34:26.764521 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.764394 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l9b9\" (UniqueName: \"kubernetes.io/projected/f00f710c-ea6e-407a-82f1-16fa9fb4a782-kube-api-access-4l9b9\") pod \"downloads-685bfb5f64-vngwn\" (UID: \"f00f710c-ea6e-407a-82f1-16fa9fb4a782\") " pod="openshift-console/downloads-685bfb5f64-vngwn" Jul 07 14:34:26.764521 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.764436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qd7pj\" (UID: \"9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj" Jul 07 14:34:26.765180 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.765132 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qd7pj\" (UID: \"9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj" Jul 07 14:34:26.768183 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.768157 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qd7pj\" (UID: \"9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj" Jul 07 14:34:26.826059 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.825970 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:26.865076 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.865040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l9b9\" (UniqueName: \"kubernetes.io/projected/f00f710c-ea6e-407a-82f1-16fa9fb4a782-kube-api-access-4l9b9\") pod \"downloads-685bfb5f64-vngwn\" (UID: \"f00f710c-ea6e-407a-82f1-16fa9fb4a782\") " pod="openshift-console/downloads-685bfb5f64-vngwn" Jul 07 14:34:26.876361 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.876331 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l9b9\" (UniqueName: \"kubernetes.io/projected/f00f710c-ea6e-407a-82f1-16fa9fb4a782-kube-api-access-4l9b9\") pod \"downloads-685bfb5f64-vngwn\" (UID: \"f00f710c-ea6e-407a-82f1-16fa9fb4a782\") " pod="openshift-console/downloads-685bfb5f64-vngwn" Jul 07 14:34:26.899086 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.899053 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj" Jul 07 14:34:26.922012 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:26.921981 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:27.008132 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:27.008077 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-vngwn" Jul 07 14:34:27.023158 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:27.023110 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:27.184184 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:27.184147 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:27.543924 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:27.543835 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:27.644266 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:27.644224 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:28.029745 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:28.029711 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:28.101597 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:28.101562 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:28.544082 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:28.544048 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:28.616182 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:28.616129 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:31.431149 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.431060 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj"] Jul 07 14:34:31.474578 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.474550 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-vngwn"] Jul 07 14:34:31.482052 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:34:31.482017 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf00f710c_ea6e_407a_82f1_16fa9fb4a782.slice/crio-6b23567888e47b38543d642670176f56337b2eac83e59e9e19d218e13a7b3228 WatchSource:0}: Error finding container 6b23567888e47b38543d642670176f56337b2eac83e59e9e19d218e13a7b3228: Status 404 returned error can't find the container with id 6b23567888e47b38543d642670176f56337b2eac83e59e9e19d218e13a7b3228 Jul 07 14:34:31.610446 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.610387 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-vngwn" event={"ID":"f00f710c-ea6e-407a-82f1-16fa9fb4a782","Type":"ContainerStarted","Data":"6b23567888e47b38543d642670176f56337b2eac83e59e9e19d218e13a7b3228"} Jul 07 14:34:31.612508 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.612475 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xb5ll" event={"ID":"91b21080-c287-47e7-9829-601c1be484f2","Type":"ContainerStarted","Data":"2e5d05d30c238ddeef407cb0f3fd1a37e3dae8ca638106d3f6a0fb8b313821f8"} Jul 07 14:34:31.617435 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.615892 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2dz7j" event={"ID":"adbb655f-b138-4258-909d-f74f718863a1","Type":"ContainerStarted","Data":"7aff79216b355f252e0db9d0933d4d7c2a930fdc51f8df1e23908898d73500d6"} Jul 07 14:34:31.620131 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.620075 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" event={"ID":"fada1bb6-9ff9-402c-b807-0a6a081e6dd0","Type":"ContainerStarted","Data":"ace410614d61d2b88f0224d107d34b4b124051e514118ead97189e7237d849bf"} Jul 07 14:34:31.625596 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.625527 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" event={"ID":"ffc93eac-e8ab-406a-96df-22c97fec6622","Type":"ContainerStarted","Data":"81fb51171ac38923eaf318b4698d4bb126f26b7d6338957a6443a1fe82a2fa96"} Jul 07 14:34:31.630053 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.630025 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gznjd" event={"ID":"8ad450ca-750b-4c5a-a5c0-bc66602536f7","Type":"ContainerStarted","Data":"8ff3f9d6e3aaca44fd950866121fcff754ba0814defce40d0bdcda2b018b6ae2"} Jul 07 14:34:31.633600 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.633355 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xb5ll" podStartSLOduration=66.985398713 podStartE2EDuration="1m15.633337496s" totalImagesPullingTime="8.647938783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:31.632914731 +0000 UTC m=+76.907998151" watchObservedRunningTime="2026-07-07 14:34:31.633337496 +0000 UTC m=+76.908420925" Jul 07 14:34:31.633711 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.633595 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj" event={"ID":"9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c","Type":"ContainerStarted","Data":"ebbd99a0ff83afb27213a0362f9faf8418d3895444771a46a2385effb2be3920"} Jul 07 14:34:31.652560 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.652506 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4c5x2" podStartSLOduration=35.42067922 podStartE2EDuration="42.652491443s" totalImagesPullingTime="7.231812223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:31.651618644 +0000 UTC m=+76.926702066" watchObservedRunningTime="2026-07-07 14:34:31.652491443 +0000 UTC m=+76.927574870" Jul 07 14:34:31.673204 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.673148 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gznjd" podStartSLOduration=17.268801791 podStartE2EDuration="28.673111915s" totalImagesPullingTime="11.404310124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:34:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:31.67112901 +0000 UTC m=+76.946212457" watchObservedRunningTime="2026-07-07 14:34:31.673111915 +0000 UTC m=+76.948195341" Jul 07 14:34:31.742325 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.742276 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:31.933625 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:31.933591 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:32.362444 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:32.362410 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:32.640734 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:32.640641 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2dz7j" event={"ID":"adbb655f-b138-4258-909d-f74f718863a1","Type":"ContainerStarted","Data":"716580ace98878625fea285cea2233a76afbfce2dc146bbd2f8db2694337a007"} Jul 07 14:34:32.641390 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:32.641333 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2dz7j" Jul 07 14:34:32.643807 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:32.643673 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" event={"ID":"fada1bb6-9ff9-402c-b807-0a6a081e6dd0","Type":"ContainerStarted","Data":"1fb366ad9ef497e97a86680c5fa617e2066ad8bee40c1d118f68564c742920bd"} Jul 07 14:34:32.664015 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:32.663858 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2dz7j" podStartSLOduration=34.425634412 podStartE2EDuration="41.663839071s" totalImagesPullingTime="7.238204659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:32.662227334 +0000 UTC m=+77.937310784" watchObservedRunningTime="2026-07-07 14:34:32.663839071 +0000 UTC m=+77.938922494" Jul 07 14:34:32.682379 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:32.681767 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zbvfq" podStartSLOduration=36.494928828 podStartE2EDuration="43.681751466s" totalImagesPullingTime="7.186822638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:32.68075782 +0000 UTC m=+77.955841269" watchObservedRunningTime="2026-07-07 14:34:32.681751466 +0000 UTC m=+77.956834894" Jul 07 14:34:33.649277 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:33.649244 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj" event={"ID":"9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c","Type":"ContainerStarted","Data":"3594c0d0263c0dbb0db2f2435008cd982955c423608addf5ca72f4c8a643f0f4"} Jul 07 14:34:33.666179 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:33.666130 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qd7pj" podStartSLOduration=5.794037843 podStartE2EDuration="7.666117632s" totalImagesPullingTime="1.872079789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:33.665264445 +0000 UTC m=+78.940347871" watchObservedRunningTime="2026-07-07 14:34:33.666117632 +0000 UTC m=+78.941201057" Jul 07 14:34:33.796642 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:33.796607 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:33.897651 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:33.897610 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:35.963535 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:35.963498 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:36.042180 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:36.042147 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:36.347969 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:36.347934 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:36.412115 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:36.412079 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:36.663331 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:36.663244 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:39.374447 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:39.374310 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:39.440640 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:39.440605 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:39.543718 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:39.543679 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:39.643847 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:39.643735 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:40.759866 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:40.759826 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:42.808792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:42.808753 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:42.902677 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:42.902641 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:43.144532 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.144499 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:43.191948 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.191910 2581 patch_prober.go:28] interesting pod/image-registry-56b97d49f8-nsl7p container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 14:34:43.192118 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.191972 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" podUID="93cb5a02-4235-47be-aedc-a5d73c3d117e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 14:34:43.311506 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.311464 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:43.435593 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.435497 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:43.488708 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.488611 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5krnj"] Jul 07 14:34:43.492735 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.492211 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.499592 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.498141 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 14:34:43.499592 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.498153 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-chw4t\"" Jul 07 14:34:43.499592 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.498272 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 14:34:43.499592 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.498459 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 14:34:43.499592 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.499320 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 14:34:43.568396 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.568367 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:43.604215 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.604179 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-wtmp\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.604376 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.604229 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a9e3c44-79ac-4e07-b853-d58ce0684394-metrics-client-ca\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.604376 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.604282 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a9e3c44-79ac-4e07-b853-d58ce0684394-sys\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.604376 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.604308 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-accelerators-collector-config\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.604554 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.604389 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4a9e3c44-79ac-4e07-b853-d58ce0684394-root\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.604554 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.604425 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-textfile\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.604554 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.604458 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zhks\" (UniqueName: \"kubernetes.io/projected/4a9e3c44-79ac-4e07-b853-d58ce0684394-kube-api-access-4zhks\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.604554 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.604477 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-tls\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.604554 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.604501 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.655637 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.655607 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2dz7j" Jul 07 14:34:43.662009 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.661986 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:43.705336 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705099 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-wtmp\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.705336 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705140 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a9e3c44-79ac-4e07-b853-d58ce0684394-metrics-client-ca\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.705336 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705170 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a9e3c44-79ac-4e07-b853-d58ce0684394-sys\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.705336 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705201 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-accelerators-collector-config\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.705336 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4a9e3c44-79ac-4e07-b853-d58ce0684394-root\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.705867 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705474 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a9e3c44-79ac-4e07-b853-d58ce0684394-sys\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.705867 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705625 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4a9e3c44-79ac-4e07-b853-d58ce0684394-root\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.705867 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705631 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-wtmp\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.706106 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705878 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-textfile\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.706106 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705899 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a9e3c44-79ac-4e07-b853-d58ce0684394-metrics-client-ca\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.706106 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705934 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zhks\" (UniqueName: \"kubernetes.io/projected/4a9e3c44-79ac-4e07-b853-d58ce0684394-kube-api-access-4zhks\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.706106 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-tls\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.706106 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705976 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-accelerators-collector-config\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.706106 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.705994 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.706507 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.706177 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-textfile\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.708946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.708909 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-tls\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.709241 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.709216 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a9e3c44-79ac-4e07-b853-d58ce0684394-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.715946 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.715582 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zhks\" (UniqueName: \"kubernetes.io/projected/4a9e3c44-79ac-4e07-b853-d58ce0684394-kube-api-access-4zhks\") pod \"node-exporter-5krnj\" (UID: \"4a9e3c44-79ac-4e07-b853-d58ce0684394\") " pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.761855 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.761810 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:43.805832 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.805785 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5krnj" Jul 07 14:34:43.817845 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:34:43.817692 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9e3c44_79ac_4e07_b853_d58ce0684394.slice/crio-52936b101d6d748be1553a8d42f60769b361696833e234a2bdf1885f3060858b WatchSource:0}: Error finding container 52936b101d6d748be1553a8d42f60769b361696833e234a2bdf1885f3060858b: Status 404 returned error can't find the container with id 52936b101d6d748be1553a8d42f60769b361696833e234a2bdf1885f3060858b Jul 07 14:34:43.862143 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:43.862110 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:44.138202 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:44.138165 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:44.399461 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:44.399384 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:44.502789 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:44.502753 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:44.699851 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:44.699668 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5krnj" event={"ID":"4a9e3c44-79ac-4e07-b853-d58ce0684394","Type":"ContainerStarted","Data":"52936b101d6d748be1553a8d42f60769b361696833e234a2bdf1885f3060858b"} Jul 07 14:34:45.059294 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:45.059201 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:45.536178 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:45.536139 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:45.588837 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:45.588794 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:34:45.616504 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:45.616468 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:46.277646 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:46.277606 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:46.383389 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:46.383285 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:46.453425 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:46.453393 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:46.523349 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:46.523319 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:47.023093 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:47.023055 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:48.175309 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:48.175270 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:48.282958 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:48.282920 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:48.548395 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:48.548287 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:48.806877 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:48.806782 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:48.908002 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:48.907955 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:49.308624 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:49.308588 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56b97d49f8-nsl7p"] Jul 07 14:34:49.414046 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:49.414001 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:49.521790 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:49.521748 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:49.591124 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:49.591095 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:49.668592 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:49.668558 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:50.399673 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:50.399626 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:50.500363 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:50.500312 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:52.412378 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:52.412343 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:52.488386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:52.488344 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:52.735501 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:52.735465 2581 generic.go:356] "Generic (PLEG): container finished" podID="4a9e3c44-79ac-4e07-b853-d58ce0684394" containerID="2a2ceb9fc6c540739534d00b564fc0d92be38e04d77826f73230db6b162dd12e" exitCode=0 Jul 07 14:34:52.735754 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:52.735513 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5krnj" event={"ID":"4a9e3c44-79ac-4e07-b853-d58ce0684394","Type":"ContainerDied","Data":"2a2ceb9fc6c540739534d00b564fc0d92be38e04d77826f73230db6b162dd12e"} Jul 07 14:34:52.737472 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:52.737437 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-vngwn" event={"ID":"f00f710c-ea6e-407a-82f1-16fa9fb4a782","Type":"ContainerStarted","Data":"962ed865778ba42f229189593a3aa679ea300b7de0d73b5d2f6526ac2a61e751"} Jul 07 14:34:52.737579 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:52.737491 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-vngwn" Jul 07 14:34:52.752361 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:52.752309 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-vngwn" Jul 07 14:34:52.775210 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:52.775150 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-vngwn" podStartSLOduration=6.549144019 podStartE2EDuration="26.775132796s" totalImagesPullingTime="20.225988777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:52.774806769 +0000 UTC m=+98.049890215" watchObservedRunningTime="2026-07-07 14:34:52.775132796 +0000 UTC m=+98.050216248" Jul 07 14:34:53.745511 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:53.745467 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5krnj" event={"ID":"4a9e3c44-79ac-4e07-b853-d58ce0684394","Type":"ContainerStarted","Data":"8dfff916650ba6564acc4388f6643d153470d3abeb2a4ad71dc6e0746a6c9f03"} Jul 07 14:34:53.745995 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:53.745542 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5krnj" event={"ID":"4a9e3c44-79ac-4e07-b853-d58ce0684394","Type":"ContainerStarted","Data":"7d022418f9d6315bdc97975b50919bce6efe80ddac0ecd089592305724f5dca5"} Jul 07 14:34:53.774460 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:53.774394 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5krnj" podStartSLOduration=2.2512322510000002 podStartE2EDuration="10.774373762s" totalImagesPullingTime="8.442115221s" totalInitContainerRuntime="81.02629ms" isStatefulPod=true podCreationTimestamp="2026-07-07 14:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:53.772637198 +0000 UTC m=+99.047720673" watchObservedRunningTime="2026-07-07 14:34:53.774373762 +0000 UTC m=+99.049457189" Jul 07 14:34:54.968174 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:54.968132 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:55.067852 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:55.067734 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:55.167562 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:55.167522 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:55.268303 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:55.268264 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:55.585738 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:55.585706 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jrxsr" Jul 07 14:34:58.569027 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:58.568966 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:58.667033 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:34:58.666993 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:01.537444 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:01.537411 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:01.637204 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:01.637169 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:04.642955 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:04.642911 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:04.743528 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:04.743497 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:05.412032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:05.412002 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:05.511396 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:05.511369 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:05.613213 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:05.613186 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:05.712642 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:05.712562 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:05.813497 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:05.813464 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:05.914347 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:05.914312 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:05.982010 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:05.981933 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:13.540092 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:13.540059 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:13.621854 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:13.621804 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:13.824895 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:13.824860 2581 generic.go:356] "Generic (PLEG): container finished" podID="615e9275-9e38-4727-8c1f-98b3e94b86e1" containerID="2e1653986a7de5f0f4c6a843a0c57b5465b56dcce212e7f87226746268570c98" exitCode=0 Jul 07 14:35:13.825059 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:13.824908 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" event={"ID":"615e9275-9e38-4727-8c1f-98b3e94b86e1","Type":"ContainerDied","Data":"2e1653986a7de5f0f4c6a843a0c57b5465b56dcce212e7f87226746268570c98"} Jul 07 14:35:13.825278 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:13.825255 2581 scope.go:117] "RemoveContainer" containerID="2e1653986a7de5f0f4c6a843a0c57b5465b56dcce212e7f87226746268570c98" Jul 07 14:35:14.339972 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.339930 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" podUID="93cb5a02-4235-47be-aedc-a5d73c3d117e" containerName="registry" containerID="cri-o://58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb" gracePeriod=30 Jul 07 14:35:14.611713 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.611642 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:35:14.671901 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.671869 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-installation-pull-secrets\") pod \"93cb5a02-4235-47be-aedc-a5d73c3d117e\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " Jul 07 14:35:14.672094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.671940 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93cb5a02-4235-47be-aedc-a5d73c3d117e-ca-trust-extracted\") pod \"93cb5a02-4235-47be-aedc-a5d73c3d117e\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " Jul 07 14:35:14.672094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.671968 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls\") pod \"93cb5a02-4235-47be-aedc-a5d73c3d117e\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " Jul 07 14:35:14.672094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.672000 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-trusted-ca\") pod \"93cb5a02-4235-47be-aedc-a5d73c3d117e\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " Jul 07 14:35:14.672094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.672040 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-image-registry-private-configuration\") pod \"93cb5a02-4235-47be-aedc-a5d73c3d117e\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " Jul 07 14:35:14.672317 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.672203 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-bound-sa-token\") pod \"93cb5a02-4235-47be-aedc-a5d73c3d117e\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " Jul 07 14:35:14.672317 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.672244 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-certificates\") pod \"93cb5a02-4235-47be-aedc-a5d73c3d117e\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " Jul 07 14:35:14.672317 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.672280 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7bgw\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-kube-api-access-r7bgw\") pod \"93cb5a02-4235-47be-aedc-a5d73c3d117e\" (UID: \"93cb5a02-4235-47be-aedc-a5d73c3d117e\") " Jul 07 14:35:14.672502 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.672471 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "93cb5a02-4235-47be-aedc-a5d73c3d117e" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:35:14.672973 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.672940 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "93cb5a02-4235-47be-aedc-a5d73c3d117e" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:35:14.674723 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.674694 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "93cb5a02-4235-47be-aedc-a5d73c3d117e" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:35:14.674723 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.674705 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "93cb5a02-4235-47be-aedc-a5d73c3d117e" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:35:14.674988 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.674778 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "93cb5a02-4235-47be-aedc-a5d73c3d117e" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:35:14.674988 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.674829 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "93cb5a02-4235-47be-aedc-a5d73c3d117e" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:35:14.674988 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.674939 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-kube-api-access-r7bgw" (OuterVolumeSpecName: "kube-api-access-r7bgw") pod "93cb5a02-4235-47be-aedc-a5d73c3d117e" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e"). InnerVolumeSpecName "kube-api-access-r7bgw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:35:14.680392 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.680371 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93cb5a02-4235-47be-aedc-a5d73c3d117e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "93cb5a02-4235-47be-aedc-a5d73c3d117e" (UID: "93cb5a02-4235-47be-aedc-a5d73c3d117e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:35:14.773633 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.773590 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93cb5a02-4235-47be-aedc-a5d73c3d117e-ca-trust-extracted\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:35:14.773633 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.773631 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-tls\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:35:14.773872 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.773649 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-trusted-ca\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:35:14.773872 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.773665 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-image-registry-private-configuration\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:35:14.773872 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.773680 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-bound-sa-token\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:35:14.773872 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.773695 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93cb5a02-4235-47be-aedc-a5d73c3d117e-registry-certificates\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:35:14.773872 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.773708 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7bgw\" (UniqueName: \"kubernetes.io/projected/93cb5a02-4235-47be-aedc-a5d73c3d117e-kube-api-access-r7bgw\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:35:14.773872 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.773723 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93cb5a02-4235-47be-aedc-a5d73c3d117e-installation-pull-secrets\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:35:14.829982 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.829949 2581 generic.go:356] "Generic (PLEG): container finished" podID="93cb5a02-4235-47be-aedc-a5d73c3d117e" containerID="58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb" exitCode=0 Jul 07 14:35:14.830164 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.829995 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" Jul 07 14:35:14.830164 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.830037 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" event={"ID":"93cb5a02-4235-47be-aedc-a5d73c3d117e","Type":"ContainerDied","Data":"58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb"} Jul 07 14:35:14.830164 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.830095 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b97d49f8-nsl7p" event={"ID":"93cb5a02-4235-47be-aedc-a5d73c3d117e","Type":"ContainerDied","Data":"c8aee38519be0b06df91d30f31865695961f272e637ae0f2adf1fbb4639207fa"} Jul 07 14:35:14.830164 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.830123 2581 scope.go:117] "RemoveContainer" containerID="58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb" Jul 07 14:35:14.832043 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.832023 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d5kgl" event={"ID":"615e9275-9e38-4727-8c1f-98b3e94b86e1","Type":"ContainerStarted","Data":"7c74ac427a51fe074c3f6993b41eb72a360db09047d49fb9fa4402bf6d0ae8be"} Jul 07 14:35:14.838845 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.838807 2581 scope.go:117] "RemoveContainer" containerID="58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb" Jul 07 14:35:14.839126 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:35:14.839087 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb\": container with ID starting with 58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb not found: ID does not exist" containerID="58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb" Jul 07 14:35:14.839200 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.839119 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb"} err="failed to get container status \"58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb\": rpc error: code = NotFound desc = could not find container \"58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb\": container with ID starting with 58504786e3bfc31bc94606a4258441c02d5a7776cd1ab980cc8b4a8ef1ca37eb not found: ID does not exist" Jul 07 14:35:14.866837 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.866754 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56b97d49f8-nsl7p"] Jul 07 14:35:14.873595 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:14.873574 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56b97d49f8-nsl7p"] Jul 07 14:35:15.241643 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:15.241552 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93cb5a02-4235-47be-aedc-a5d73c3d117e" path="/var/lib/kubelet/pods/93cb5a02-4235-47be-aedc-a5d73c3d117e/volumes" Jul 07 14:35:27.878045 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:27.878014 2581 generic.go:356] "Generic (PLEG): container finished" podID="dd9bcdd4-6d7b-4809-befd-7238b2ceb035" containerID="cede7d66a7df4e7434dc3ea234fad250658e7545166368ad967fa0846ebfdcdc" exitCode=0 Jul 07 14:35:27.878467 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:27.878090 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" event={"ID":"dd9bcdd4-6d7b-4809-befd-7238b2ceb035","Type":"ContainerDied","Data":"cede7d66a7df4e7434dc3ea234fad250658e7545166368ad967fa0846ebfdcdc"} Jul 07 14:35:27.878467 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:27.878401 2581 scope.go:117] "RemoveContainer" containerID="cede7d66a7df4e7434dc3ea234fad250658e7545166368ad967fa0846ebfdcdc" Jul 07 14:35:28.549747 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:28.549713 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:28.614584 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:28.614550 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:28.890363 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:28.890323 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-gwq7g" event={"ID":"dd9bcdd4-6d7b-4809-befd-7238b2ceb035","Type":"ContainerStarted","Data":"b87291fb5c5ac90078fb6b8013eabb377f647a9d8db3aa65a52c993432e1f165"} Jul 07 14:35:33.909797 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:33.909765 2581 generic.go:356] "Generic (PLEG): container finished" podID="27bf9b12-515c-49fd-902c-9ca037f5fa38" containerID="c55a9ed943c9cb542dc4db7d2ab83093a2e463cb24adc1f4b7ffa7339499ca55" exitCode=0 Jul 07 14:35:33.910266 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:33.909846 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" event={"ID":"27bf9b12-515c-49fd-902c-9ca037f5fa38","Type":"ContainerDied","Data":"c55a9ed943c9cb542dc4db7d2ab83093a2e463cb24adc1f4b7ffa7339499ca55"} Jul 07 14:35:33.910266 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:33.910185 2581 scope.go:117] "RemoveContainer" containerID="c55a9ed943c9cb542dc4db7d2ab83093a2e463cb24adc1f4b7ffa7339499ca55" Jul 07 14:35:34.915225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:34.915193 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-lqck8" event={"ID":"27bf9b12-515c-49fd-902c-9ca037f5fa38","Type":"ContainerStarted","Data":"6f086a40f00a120f61f2301c9a6fda2896595e4a7de1979aa7c8dd4c92644be8"} Jul 07 14:35:37.051085 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:37.051045 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:37.153386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:37.153352 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:59.764269 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:59.764222 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:59.841030 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:35:59.840998 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:11.023307 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:11.023196 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:11.162905 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:11.162871 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:11.235109 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:11.235081 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:11.482116 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:11.482074 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:11.599442 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:11.599407 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:11.752238 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:11.752146 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:11.854144 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:11.854116 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:21.973226 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:21.973184 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:22.074246 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:22.074214 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:22.133049 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:22.133015 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:28.549848 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:28.549778 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:28.644218 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:28.644191 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:34.102487 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:34.102454 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:34.237974 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:34.237943 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:51.922145 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:51.922096 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:52.015024 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.014991 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:52.108851 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.108810 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:52.116584 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.116556 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g"] Jul 07 14:36:52.116838 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.116813 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93cb5a02-4235-47be-aedc-a5d73c3d117e" containerName="registry" Jul 07 14:36:52.116888 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.116840 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="93cb5a02-4235-47be-aedc-a5d73c3d117e" containerName="registry" Jul 07 14:36:52.116923 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.116899 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="93cb5a02-4235-47be-aedc-a5d73c3d117e" containerName="registry" Jul 07 14:36:52.118790 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.118773 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:52.121710 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.121686 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 14:36:52.121840 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.121771 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 14:36:52.122094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.122076 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 14:36:52.122385 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.122358 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 14:36:52.132409 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.132380 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g"] Jul 07 14:36:52.218878 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.218804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ld2r\" (UniqueName: \"kubernetes.io/projected/76038b78-4cbd-4729-adb1-4c528412363f-kube-api-access-6ld2r\") pod \"klusterlet-addon-workmgr-678b4c95c6-l959g\" (UID: \"76038b78-4cbd-4729-adb1-4c528412363f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:52.218878 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.218854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/76038b78-4cbd-4729-adb1-4c528412363f-klusterlet-config\") pod \"klusterlet-addon-workmgr-678b4c95c6-l959g\" (UID: \"76038b78-4cbd-4729-adb1-4c528412363f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:52.219008 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.218897 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76038b78-4cbd-4729-adb1-4c528412363f-tmp\") pod \"klusterlet-addon-workmgr-678b4c95c6-l959g\" (UID: \"76038b78-4cbd-4729-adb1-4c528412363f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:52.319509 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.319486 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ld2r\" (UniqueName: \"kubernetes.io/projected/76038b78-4cbd-4729-adb1-4c528412363f-kube-api-access-6ld2r\") pod \"klusterlet-addon-workmgr-678b4c95c6-l959g\" (UID: \"76038b78-4cbd-4729-adb1-4c528412363f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:52.319592 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.319516 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/76038b78-4cbd-4729-adb1-4c528412363f-klusterlet-config\") pod \"klusterlet-addon-workmgr-678b4c95c6-l959g\" (UID: \"76038b78-4cbd-4729-adb1-4c528412363f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:52.319592 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.319548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76038b78-4cbd-4729-adb1-4c528412363f-tmp\") pod \"klusterlet-addon-workmgr-678b4c95c6-l959g\" (UID: \"76038b78-4cbd-4729-adb1-4c528412363f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:52.319904 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.319881 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76038b78-4cbd-4729-adb1-4c528412363f-tmp\") pod \"klusterlet-addon-workmgr-678b4c95c6-l959g\" (UID: \"76038b78-4cbd-4729-adb1-4c528412363f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:52.321901 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.321885 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/76038b78-4cbd-4729-adb1-4c528412363f-klusterlet-config\") pod \"klusterlet-addon-workmgr-678b4c95c6-l959g\" (UID: \"76038b78-4cbd-4729-adb1-4c528412363f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:52.328995 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.328978 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ld2r\" (UniqueName: \"kubernetes.io/projected/76038b78-4cbd-4729-adb1-4c528412363f-kube-api-access-6ld2r\") pod \"klusterlet-addon-workmgr-678b4c95c6-l959g\" (UID: \"76038b78-4cbd-4729-adb1-4c528412363f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:52.427678 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.427646 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:52.552874 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:52.552849 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g"] Jul 07 14:36:52.555726 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:36:52.555696 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76038b78_4cbd_4729_adb1_4c528412363f.slice/crio-7ea3120a4cde442e2f1c58fa6bb02effb7876225739c909fe7cfdca4cae90a66 WatchSource:0}: Error finding container 7ea3120a4cde442e2f1c58fa6bb02effb7876225739c909fe7cfdca4cae90a66: Status 404 returned error can't find the container with id 7ea3120a4cde442e2f1c58fa6bb02effb7876225739c909fe7cfdca4cae90a66 Jul 07 14:36:53.202428 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:53.202396 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" event={"ID":"76038b78-4cbd-4729-adb1-4c528412363f","Type":"ContainerStarted","Data":"7ea3120a4cde442e2f1c58fa6bb02effb7876225739c909fe7cfdca4cae90a66"} Jul 07 14:36:57.226690 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:57.226655 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" event={"ID":"76038b78-4cbd-4729-adb1-4c528412363f","Type":"ContainerStarted","Data":"a29c243964e60e4ae364880203603f1360e08933da8f94924279f9e6690b8994"} Jul 07 14:36:57.227110 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:57.226894 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:57.228469 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:57.228447 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" Jul 07 14:36:57.245811 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:57.245760 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678b4c95c6-l959g" podStartSLOduration=1.5105704709999999 podStartE2EDuration="5.245747721s" totalImagesPullingTime="3.73517725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:36:57.244100809 +0000 UTC m=+222.519184235" watchObservedRunningTime="2026-07-07 14:36:57.245747721 +0000 UTC m=+222.520831147" Jul 07 14:36:59.104990 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:59.104947 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:59.412086 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:36:59.411995 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:03.135129 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:03.135096 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:03.238501 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:03.238471 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:20.303026 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:20.302984 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:29.999713 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:29.999682 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:30.850095 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:30.850064 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:31.962395 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:31.962366 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8"] Jul 07 14:37:31.966810 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:31.966793 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8" Jul 07 14:37:31.969444 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:31.969422 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:37:31.969557 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:31.969513 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7hqc4\"" Jul 07 14:37:31.969707 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:31.969688 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 14:37:31.980835 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:31.978098 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8"] Jul 07 14:37:32.098890 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:32.098858 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhdxj\" (UniqueName: \"kubernetes.io/projected/83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b-kube-api-access-lhdxj\") pod \"cert-manager-operator-controller-manager-55c444bf86-slgd8\" (UID: \"83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8" Jul 07 14:37:32.099033 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:32.098908 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-slgd8\" (UID: \"83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8" Jul 07 14:37:32.199727 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:32.199688 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhdxj\" (UniqueName: \"kubernetes.io/projected/83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b-kube-api-access-lhdxj\") pod \"cert-manager-operator-controller-manager-55c444bf86-slgd8\" (UID: \"83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8" Jul 07 14:37:32.199895 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:32.199740 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-slgd8\" (UID: \"83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8" Jul 07 14:37:32.200101 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:32.200084 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-slgd8\" (UID: \"83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8" Jul 07 14:37:32.207765 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:32.207746 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhdxj\" (UniqueName: \"kubernetes.io/projected/83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b-kube-api-access-lhdxj\") pod \"cert-manager-operator-controller-manager-55c444bf86-slgd8\" (UID: \"83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8" Jul 07 14:37:32.276244 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:32.276177 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8" Jul 07 14:37:32.405558 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:32.405522 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8"] Jul 07 14:37:32.409635 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:37:32.409606 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83d9cb32_45a5_4bf9_ba64_bc34fecaeb1b.slice/crio-ebb1f21d9e27963ca3594ffe1caead5a0877e6400f3ebd7870dabe818dc781ff WatchSource:0}: Error finding container ebb1f21d9e27963ca3594ffe1caead5a0877e6400f3ebd7870dabe818dc781ff: Status 404 returned error can't find the container with id ebb1f21d9e27963ca3594ffe1caead5a0877e6400f3ebd7870dabe818dc781ff Jul 07 14:37:33.370691 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:33.370657 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8" event={"ID":"83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b","Type":"ContainerStarted","Data":"ebb1f21d9e27963ca3594ffe1caead5a0877e6400f3ebd7870dabe818dc781ff"} Jul 07 14:37:33.624626 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:33.624553 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:33.724353 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:33.724314 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:35.381040 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:35.381007 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8" event={"ID":"83d9cb32-45a5-4bf9-ba64-bc34fecaeb1b","Type":"ContainerStarted","Data":"34cb9f259facc79157e416606f63aba62bbc74423c2cc12ace799a10db2ed6d5"} Jul 07 14:37:35.401696 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:35.401650 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-slgd8" podStartSLOduration=1.581442283 podStartE2EDuration="4.401634537s" totalImagesPullingTime="2.820192254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:37:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:37:35.399179425 +0000 UTC m=+260.674262847" watchObservedRunningTime="2026-07-07 14:37:35.401634537 +0000 UTC m=+260.676717962" Jul 07 14:37:35.793182 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:35.793144 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:35.902812 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:35.902781 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:36.230171 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:36.230138 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:36.324696 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:36.324662 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:38.610065 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:38.609989 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:38.832278 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:38.832244 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:40.610063 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:40.610026 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:40.663740 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:40.663713 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:46.236000 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:46.235965 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:49.062769 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.062725 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qr9gg"] Jul 07 14:37:49.066097 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.066076 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qr9gg" Jul 07 14:37:49.068961 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.068934 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 14:37:49.069091 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.068942 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-scsb5\"" Jul 07 14:37:49.069781 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.069757 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 14:37:49.075540 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.075513 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qr9gg"] Jul 07 14:37:49.217667 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.217633 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9mgx\" (UniqueName: \"kubernetes.io/projected/6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7-kube-api-access-f9mgx\") pod \"cert-manager-55b4d954d5-qr9gg\" (UID: \"6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7\") " pod="cert-manager/cert-manager-55b4d954d5-qr9gg" Jul 07 14:37:49.217783 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.217684 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7-bound-sa-token\") pod \"cert-manager-55b4d954d5-qr9gg\" (UID: \"6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7\") " pod="cert-manager/cert-manager-55b4d954d5-qr9gg" Jul 07 14:37:49.319106 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.319026 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9mgx\" (UniqueName: \"kubernetes.io/projected/6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7-kube-api-access-f9mgx\") pod \"cert-manager-55b4d954d5-qr9gg\" (UID: \"6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7\") " pod="cert-manager/cert-manager-55b4d954d5-qr9gg" Jul 07 14:37:49.319106 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.319090 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7-bound-sa-token\") pod \"cert-manager-55b4d954d5-qr9gg\" (UID: \"6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7\") " pod="cert-manager/cert-manager-55b4d954d5-qr9gg" Jul 07 14:37:49.327836 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.327781 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7-bound-sa-token\") pod \"cert-manager-55b4d954d5-qr9gg\" (UID: \"6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7\") " pod="cert-manager/cert-manager-55b4d954d5-qr9gg" Jul 07 14:37:49.327836 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.327804 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9mgx\" (UniqueName: \"kubernetes.io/projected/6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7-kube-api-access-f9mgx\") pod \"cert-manager-55b4d954d5-qr9gg\" (UID: \"6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7\") " pod="cert-manager/cert-manager-55b4d954d5-qr9gg" Jul 07 14:37:49.384804 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.384774 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qr9gg" Jul 07 14:37:49.513301 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:49.513267 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qr9gg"] Jul 07 14:37:49.516118 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:37:49.516090 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e74c0f7_e87a_464c_8eb2_8d4cc81c07f7.slice/crio-0b4b1174f38fdb2693fb5c49b82957991db6efa00578a07ae2481d1d7ae7429f WatchSource:0}: Error finding container 0b4b1174f38fdb2693fb5c49b82957991db6efa00578a07ae2481d1d7ae7429f: Status 404 returned error can't find the container with id 0b4b1174f38fdb2693fb5c49b82957991db6efa00578a07ae2481d1d7ae7429f Jul 07 14:37:50.453405 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:50.453369 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qr9gg" event={"ID":"6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7","Type":"ContainerStarted","Data":"0b4b1174f38fdb2693fb5c49b82957991db6efa00578a07ae2481d1d7ae7429f"} Jul 07 14:37:51.608529 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:51.608471 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:51.709847 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:51.709795 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:52.463618 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:52.463584 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qr9gg" event={"ID":"6e74c0f7-e87a-464c-8eb2-8d4cc81c07f7","Type":"ContainerStarted","Data":"da682975fea30116d3d824857a6f6d43facb53192a401184edfc100e963e17ca"} Jul 07 14:37:52.480252 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:52.480204 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-qr9gg" podStartSLOduration=1.139666744 podStartE2EDuration="3.48018888s" totalImagesPullingTime="2.340522136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:37:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:37:52.478721434 +0000 UTC m=+277.753804862" watchObservedRunningTime="2026-07-07 14:37:52.48018888 +0000 UTC m=+277.755272306" Jul 07 14:37:57.979206 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:57.979139 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:58.078570 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:58.078538 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:58.871122 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:58.871085 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:58.971711 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:37:58.971678 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:00.171149 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:00.171121 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:12.627103 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:12.627076 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:12.731927 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:12.731901 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:13.003102 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.003043 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:13.106313 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.106285 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:13.132487 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.132459 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw"] Jul 07 14:38:13.138380 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.138354 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.140731 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.140715 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 14:38:13.141084 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.141058 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 14:38:13.141149 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.141130 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fwtsc\"" Jul 07 14:38:13.141325 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.141310 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 14:38:13.142371 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.142350 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 14:38:13.161521 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.156952 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw"] Jul 07 14:38:13.188165 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.188144 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d06cd06-e75a-49ff-8c91-f36581f6efac-tmp\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.188262 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.188182 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6d06cd06-e75a-49ff-8c91-f36581f6efac-manifests\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.188262 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.188201 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6d06cd06-e75a-49ff-8c91-f36581f6efac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.188262 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.188230 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d06cd06-e75a-49ff-8c91-f36581f6efac-webhook-cert\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.188367 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.188261 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vv8q\" (UniqueName: \"kubernetes.io/projected/6d06cd06-e75a-49ff-8c91-f36581f6efac-kube-api-access-5vv8q\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.288878 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.288807 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6d06cd06-e75a-49ff-8c91-f36581f6efac-manifests\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.288878 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.288866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6d06cd06-e75a-49ff-8c91-f36581f6efac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.289014 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.288917 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d06cd06-e75a-49ff-8c91-f36581f6efac-webhook-cert\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.289014 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.288942 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vv8q\" (UniqueName: \"kubernetes.io/projected/6d06cd06-e75a-49ff-8c91-f36581f6efac-kube-api-access-5vv8q\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.289014 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.289003 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d06cd06-e75a-49ff-8c91-f36581f6efac-tmp\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.289179 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.289156 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6d06cd06-e75a-49ff-8c91-f36581f6efac-manifests\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.291138 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.291118 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d06cd06-e75a-49ff-8c91-f36581f6efac-tmp\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.291332 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.291310 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d06cd06-e75a-49ff-8c91-f36581f6efac-webhook-cert\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.291396 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.291382 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6d06cd06-e75a-49ff-8c91-f36581f6efac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.302600 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.302582 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vv8q\" (UniqueName: \"kubernetes.io/projected/6d06cd06-e75a-49ff-8c91-f36581f6efac-kube-api-access-5vv8q\") pod \"opendatahub-operator-controller-manager-cd69c54fc-gdlmw\" (UID: \"6d06cd06-e75a-49ff-8c91-f36581f6efac\") " pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.320617 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.320593 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:13.433314 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.433290 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:13.449268 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.449249 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:13.521555 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.521531 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:13.584129 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.584105 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw"] Jul 07 14:38:13.586867 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:38:13.586842 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d06cd06_e75a_49ff_8c91_f36581f6efac.slice/crio-995de9acd17ac84991f143e96a683708dd72604ba568beebf60bc4895f2a3a4d WatchSource:0}: Error finding container 995de9acd17ac84991f143e96a683708dd72604ba568beebf60bc4895f2a3a4d: Status 404 returned error can't find the container with id 995de9acd17ac84991f143e96a683708dd72604ba568beebf60bc4895f2a3a4d Jul 07 14:38:13.605863 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.605840 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:13.677272 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.677244 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:13.778014 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:13.777992 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:14.511937 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:14.511908 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:14.554442 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:14.554406 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" event={"ID":"6d06cd06-e75a-49ff-8c91-f36581f6efac","Type":"ContainerStarted","Data":"995de9acd17ac84991f143e96a683708dd72604ba568beebf60bc4895f2a3a4d"} Jul 07 14:38:14.622036 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:14.622008 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:15.115493 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:15.115461 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6rwzn_88bef153-2452-4b8c-84f4-333d8426694d/console-operator/1.log" Jul 07 14:38:15.120310 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:15.120282 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6rwzn_88bef153-2452-4b8c-84f4-333d8426694d/console-operator/1.log" Jul 07 14:38:15.139245 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:15.139223 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 14:38:17.570569 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:17.570534 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" event={"ID":"6d06cd06-e75a-49ff-8c91-f36581f6efac","Type":"ContainerStarted","Data":"0161b49ebaba0cb82e4a03ef9ed831e5aaf7723a24ec9788d1a822a21b07f81c"} Jul 07 14:38:18.575659 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:18.575590 2581 generic.go:356] "Generic (PLEG): container finished" podID="6d06cd06-e75a-49ff-8c91-f36581f6efac" containerID="0161b49ebaba0cb82e4a03ef9ed831e5aaf7723a24ec9788d1a822a21b07f81c" exitCode=0 Jul 07 14:38:18.575659 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:18.575630 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" event={"ID":"6d06cd06-e75a-49ff-8c91-f36581f6efac","Type":"ContainerDied","Data":"0161b49ebaba0cb82e4a03ef9ed831e5aaf7723a24ec9788d1a822a21b07f81c"} Jul 07 14:38:18.576114 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:18.576098 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:38:19.581301 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:19.581272 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" event={"ID":"6d06cd06-e75a-49ff-8c91-f36581f6efac","Type":"ContainerStarted","Data":"934a4ef2788a7e1017d56d7616143671f63d9bba86d1db83a2ff04147e3a11d7"} Jul 07 14:38:19.581742 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:19.581361 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:19.604095 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:19.604041 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" podStartSLOduration=1.716301615 podStartE2EDuration="6.604023974s" totalImagesPullingTime="3.87977897s" totalInitContainerRuntime="1.007943389s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:38:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:38:19.603709065 +0000 UTC m=+304.878792507" watchObservedRunningTime="2026-07-07 14:38:19.604023974 +0000 UTC m=+304.879107400" Jul 07 14:38:23.572928 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:23.572874 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:24.885884 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:24.885854 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:28.740963 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:28.740931 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:28.801304 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:28.801277 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:30.587594 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:30.587567 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-cd69c54fc-gdlmw" Jul 07 14:38:33.148493 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:33.148446 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:33.267383 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:33.267353 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:34.042273 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.042243 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:34.155871 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.155841 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp"] Jul 07 14:38:34.161195 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.161165 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" Jul 07 14:38:34.164394 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.164372 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:34.165697 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.165677 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gzm8j\"" Jul 07 14:38:34.165791 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.165712 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 14:38:34.165791 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.165717 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 14:38:34.172930 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.172911 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp"] Jul 07 14:38:34.242622 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.242598 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e2c220f-9325-4a4e-bd79-38ec5bf729c5-tmp\") pod \"kube-auth-proxy-54bbbc8986-7sghp\" (UID: \"1e2c220f-9325-4a4e-bd79-38ec5bf729c5\") " pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" Jul 07 14:38:34.242721 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.242655 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e2c220f-9325-4a4e-bd79-38ec5bf729c5-tls-certs\") pod \"kube-auth-proxy-54bbbc8986-7sghp\" (UID: \"1e2c220f-9325-4a4e-bd79-38ec5bf729c5\") " pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" Jul 07 14:38:34.242721 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.242704 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6mmk\" (UniqueName: \"kubernetes.io/projected/1e2c220f-9325-4a4e-bd79-38ec5bf729c5-kube-api-access-q6mmk\") pod \"kube-auth-proxy-54bbbc8986-7sghp\" (UID: \"1e2c220f-9325-4a4e-bd79-38ec5bf729c5\") " pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" Jul 07 14:38:34.287425 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.287406 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:34.343978 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.343945 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e2c220f-9325-4a4e-bd79-38ec5bf729c5-tls-certs\") pod \"kube-auth-proxy-54bbbc8986-7sghp\" (UID: \"1e2c220f-9325-4a4e-bd79-38ec5bf729c5\") " pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" Jul 07 14:38:34.344156 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.344018 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6mmk\" (UniqueName: \"kubernetes.io/projected/1e2c220f-9325-4a4e-bd79-38ec5bf729c5-kube-api-access-q6mmk\") pod \"kube-auth-proxy-54bbbc8986-7sghp\" (UID: \"1e2c220f-9325-4a4e-bd79-38ec5bf729c5\") " pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" Jul 07 14:38:34.344156 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.344054 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e2c220f-9325-4a4e-bd79-38ec5bf729c5-tmp\") pod \"kube-auth-proxy-54bbbc8986-7sghp\" (UID: \"1e2c220f-9325-4a4e-bd79-38ec5bf729c5\") " pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" Jul 07 14:38:34.346173 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.346153 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e2c220f-9325-4a4e-bd79-38ec5bf729c5-tmp\") pod \"kube-auth-proxy-54bbbc8986-7sghp\" (UID: \"1e2c220f-9325-4a4e-bd79-38ec5bf729c5\") " pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" Jul 07 14:38:34.346298 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.346281 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e2c220f-9325-4a4e-bd79-38ec5bf729c5-tls-certs\") pod \"kube-auth-proxy-54bbbc8986-7sghp\" (UID: \"1e2c220f-9325-4a4e-bd79-38ec5bf729c5\") " pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" Jul 07 14:38:34.354557 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.354538 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6mmk\" (UniqueName: \"kubernetes.io/projected/1e2c220f-9325-4a4e-bd79-38ec5bf729c5-kube-api-access-q6mmk\") pod \"kube-auth-proxy-54bbbc8986-7sghp\" (UID: \"1e2c220f-9325-4a4e-bd79-38ec5bf729c5\") " pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" Jul 07 14:38:34.470991 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.470967 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" Jul 07 14:38:34.614567 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.614498 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp"] Jul 07 14:38:34.617366 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:38:34.617337 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e2c220f_9325_4a4e_bd79_38ec5bf729c5.slice/crio-5f1c1ddf1f3964f037a488c7919a7b028aa4839015425c33a0aa367a4584a9b3 WatchSource:0}: Error finding container 5f1c1ddf1f3964f037a488c7919a7b028aa4839015425c33a0aa367a4584a9b3: Status 404 returned error can't find the container with id 5f1c1ddf1f3964f037a488c7919a7b028aa4839015425c33a0aa367a4584a9b3 Jul 07 14:38:34.640348 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.640322 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" event={"ID":"1e2c220f-9325-4a4e-bd79-38ec5bf729c5","Type":"ContainerStarted","Data":"5f1c1ddf1f3964f037a488c7919a7b028aa4839015425c33a0aa367a4584a9b3"} Jul 07 14:38:34.917565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.917415 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:34.960806 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.960776 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6hsjf"] Jul 07 14:38:34.964301 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.964283 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" Jul 07 14:38:34.967204 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.967180 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 07 14:38:34.967291 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.967257 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-kbvxx\"" Jul 07 14:38:34.976735 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:34.976716 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6hsjf"] Jul 07 14:38:35.021865 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.021844 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:35.051145 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.051120 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9t6d\" (UniqueName: \"kubernetes.io/projected/317793be-a149-4f2e-84c8-699683a38292-kube-api-access-k9t6d\") pod \"model-serving-api-54d89bd5fc-6hsjf\" (UID: \"317793be-a149-4f2e-84c8-699683a38292\") " pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" Jul 07 14:38:35.051239 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.051173 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/317793be-a149-4f2e-84c8-699683a38292-tls-certs\") pod \"model-serving-api-54d89bd5fc-6hsjf\" (UID: \"317793be-a149-4f2e-84c8-699683a38292\") " pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" Jul 07 14:38:35.115039 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.115001 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:35.152181 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.152159 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9t6d\" (UniqueName: \"kubernetes.io/projected/317793be-a149-4f2e-84c8-699683a38292-kube-api-access-k9t6d\") pod \"model-serving-api-54d89bd5fc-6hsjf\" (UID: \"317793be-a149-4f2e-84c8-699683a38292\") " pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" Jul 07 14:38:35.152279 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.152203 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/317793be-a149-4f2e-84c8-699683a38292-tls-certs\") pod \"model-serving-api-54d89bd5fc-6hsjf\" (UID: \"317793be-a149-4f2e-84c8-699683a38292\") " pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" Jul 07 14:38:35.154522 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.154505 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/317793be-a149-4f2e-84c8-699683a38292-tls-certs\") pod \"model-serving-api-54d89bd5fc-6hsjf\" (UID: \"317793be-a149-4f2e-84c8-699683a38292\") " pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" Jul 07 14:38:35.166551 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.166529 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9t6d\" (UniqueName: \"kubernetes.io/projected/317793be-a149-4f2e-84c8-699683a38292-kube-api-access-k9t6d\") pod \"model-serving-api-54d89bd5fc-6hsjf\" (UID: \"317793be-a149-4f2e-84c8-699683a38292\") " pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" Jul 07 14:38:35.274730 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.274675 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" Jul 07 14:38:35.400724 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.400691 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6hsjf"] Jul 07 14:38:35.403533 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:38:35.403509 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod317793be_a149_4f2e_84c8_699683a38292.slice/crio-4dfa92f6f0586dc0f2040282682f2160d80caca61b8e155c05f906abab359e0b WatchSource:0}: Error finding container 4dfa92f6f0586dc0f2040282682f2160d80caca61b8e155c05f906abab359e0b: Status 404 returned error can't find the container with id 4dfa92f6f0586dc0f2040282682f2160d80caca61b8e155c05f906abab359e0b Jul 07 14:38:35.552449 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.552380 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-spw6v"] Jul 07 14:38:35.555223 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.555202 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.555696 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.555656 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:35.557854 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.557836 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 14:38:35.558124 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.558107 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 14:38:35.558380 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.558367 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 14:38:35.558444 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.558401 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 14:38:35.558444 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.558413 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-p6gjh\"" Jul 07 14:38:35.571878 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.571855 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-spw6v"] Jul 07 14:38:35.629457 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.629422 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:35.645777 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.645748 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" event={"ID":"317793be-a149-4f2e-84c8-699683a38292","Type":"ContainerStarted","Data":"4dfa92f6f0586dc0f2040282682f2160d80caca61b8e155c05f906abab359e0b"} Jul 07 14:38:35.656246 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.656221 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-local-certs\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.656318 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.656258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.656318 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.656284 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-token\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.656318 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.656303 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.656454 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.656369 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds95p\" (UniqueName: \"kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-kube-api-access-ds95p\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.656454 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.656398 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-cacerts\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.656454 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.656423 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.757593 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.757569 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-cacerts\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.757725 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.757607 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.757725 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.757652 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-local-certs\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.757725 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.757688 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.757725 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.757717 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-token\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.758002 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.757739 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.758002 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.757780 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds95p\" (UniqueName: \"kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-kube-api-access-ds95p\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.758642 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.758610 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.760509 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.760446 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.760613 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.760511 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.760613 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.760565 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-cacerts\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.761312 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.761292 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-local-certs\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.766709 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.766684 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds95p\" (UniqueName: \"kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-kube-api-access-ds95p\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.767230 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.767211 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-token\") pod \"istiod-openshift-gateway-7574459656-spw6v\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:35.865529 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:35.865503 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:36.008979 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:36.008956 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-spw6v"] Jul 07 14:38:36.011228 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:38:36.011198 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6b66eb6_85c0_4e19_941f_3d84bae9a2c4.slice/crio-0ff26e26bac14d33dc9bd84185548cf123085268d84fd3389ddf8172cde34212 WatchSource:0}: Error finding container 0ff26e26bac14d33dc9bd84185548cf123085268d84fd3389ddf8172cde34212: Status 404 returned error can't find the container with id 0ff26e26bac14d33dc9bd84185548cf123085268d84fd3389ddf8172cde34212 Jul 07 14:38:36.654087 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:36.654006 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" event={"ID":"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4","Type":"ContainerStarted","Data":"0ff26e26bac14d33dc9bd84185548cf123085268d84fd3389ddf8172cde34212"} Jul 07 14:38:36.699075 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:36.699030 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:38.873131 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:38.873097 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 14:38:38.873493 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:38.873178 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 14:38:39.659296 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.659259 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:39.672435 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.672407 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" event={"ID":"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4","Type":"ContainerStarted","Data":"35041333ab3abe6b8e16365d64da126120496ec7ce791705e790160f8a5e5540"} Jul 07 14:38:39.672668 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.672650 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:39.674412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.674384 2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-spw6v container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 14:38:39.674529 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.674464 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" podUID="f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 14:38:39.674627 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.674603 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" event={"ID":"1e2c220f-9325-4a4e-bd79-38ec5bf729c5","Type":"ContainerStarted","Data":"05e471fe371b386bf848f4b135ec763cb57264b251899dbc309248efc66cd2f7"} Jul 07 14:38:39.676298 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.676272 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" event={"ID":"317793be-a149-4f2e-84c8-699683a38292","Type":"ContainerStarted","Data":"3af1a81798da8d2176a8e6496aa4288777bb500fc66a4d1de5cc7d660c7a23f1"} Jul 07 14:38:39.676460 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.676444 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" Jul 07 14:38:39.701584 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.701539 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" podStartSLOduration=1.841743937 podStartE2EDuration="4.701525774s" totalImagesPullingTime="2.859781837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:38:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:38:39.698976752 +0000 UTC m=+324.974060204" watchObservedRunningTime="2026-07-07 14:38:39.701525774 +0000 UTC m=+324.976609199" Jul 07 14:38:39.720690 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.720641 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-54bbbc8986-7sghp" podStartSLOduration=1.46917989 podStartE2EDuration="5.72062417s" totalImagesPullingTime="4.25144428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:38:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:38:39.719281993 +0000 UTC m=+324.994365450" watchObservedRunningTime="2026-07-07 14:38:39.72062417 +0000 UTC m=+324.995707597" Jul 07 14:38:39.738991 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.738938 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" podStartSLOduration=2.277525712 podStartE2EDuration="5.738923724s" totalImagesPullingTime="3.461398012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:38:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:38:39.738681014 +0000 UTC m=+325.013764457" watchObservedRunningTime="2026-07-07 14:38:39.738923724 +0000 UTC m=+325.014007151" Jul 07 14:38:39.746299 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:39.746262 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:40.356330 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:40.356291 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:40.681456 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:40.681375 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:40.707927 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:40.707899 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:40.782735 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:40.782709 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:45.561450 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.561413 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:45.647124 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.647089 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mx9js"] Jul 07 14:38:45.651640 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.651607 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt"] Jul 07 14:38:45.657379 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.657354 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" Jul 07 14:38:45.657716 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.657685 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" Jul 07 14:38:45.661100 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.661077 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-z6n4j\"" Jul 07 14:38:45.662183 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.662160 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-zhpdm\"" Jul 07 14:38:45.662281 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.662199 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 14:38:45.662281 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.662214 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 14:38:45.669716 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.669693 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mx9js"] Jul 07 14:38:45.671452 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.671431 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt"] Jul 07 14:38:45.674410 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.674390 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:45.744050 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.743996 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z42h\" (UniqueName: \"kubernetes.io/projected/9b8f1b96-0570-4f69-a62e-940496a1b239-kube-api-access-7z42h\") pod \"llmisvc-controller-manager-777c6bd7d5-64zpt\" (UID: \"9b8f1b96-0570-4f69-a62e-940496a1b239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" Jul 07 14:38:45.744255 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.744133 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/130e44c8-ac80-4545-a965-67b82988c0a8-cert\") pod \"kserve-controller-manager-d6f4b65b8-mx9js\" (UID: \"130e44c8-ac80-4545-a965-67b82988c0a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" Jul 07 14:38:45.744255 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.744161 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4fsz\" (UniqueName: \"kubernetes.io/projected/130e44c8-ac80-4545-a965-67b82988c0a8-kube-api-access-q4fsz\") pod \"kserve-controller-manager-d6f4b65b8-mx9js\" (UID: \"130e44c8-ac80-4545-a965-67b82988c0a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" Jul 07 14:38:45.744255 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.744181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b8f1b96-0570-4f69-a62e-940496a1b239-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-64zpt\" (UID: \"9b8f1b96-0570-4f69-a62e-940496a1b239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" Jul 07 14:38:45.806993 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.806964 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:45.844810 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.844784 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z42h\" (UniqueName: \"kubernetes.io/projected/9b8f1b96-0570-4f69-a62e-940496a1b239-kube-api-access-7z42h\") pod \"llmisvc-controller-manager-777c6bd7d5-64zpt\" (UID: \"9b8f1b96-0570-4f69-a62e-940496a1b239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" Jul 07 14:38:45.844937 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.844868 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/130e44c8-ac80-4545-a965-67b82988c0a8-cert\") pod \"kserve-controller-manager-d6f4b65b8-mx9js\" (UID: \"130e44c8-ac80-4545-a965-67b82988c0a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" Jul 07 14:38:45.844937 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.844890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4fsz\" (UniqueName: \"kubernetes.io/projected/130e44c8-ac80-4545-a965-67b82988c0a8-kube-api-access-q4fsz\") pod \"kserve-controller-manager-d6f4b65b8-mx9js\" (UID: \"130e44c8-ac80-4545-a965-67b82988c0a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" Jul 07 14:38:45.844937 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.844907 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b8f1b96-0570-4f69-a62e-940496a1b239-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-64zpt\" (UID: \"9b8f1b96-0570-4f69-a62e-940496a1b239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" Jul 07 14:38:45.847366 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.847345 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/130e44c8-ac80-4545-a965-67b82988c0a8-cert\") pod \"kserve-controller-manager-d6f4b65b8-mx9js\" (UID: \"130e44c8-ac80-4545-a965-67b82988c0a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" Jul 07 14:38:45.847478 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.847456 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b8f1b96-0570-4f69-a62e-940496a1b239-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-64zpt\" (UID: \"9b8f1b96-0570-4f69-a62e-940496a1b239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" Jul 07 14:38:45.862207 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.862186 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z42h\" (UniqueName: \"kubernetes.io/projected/9b8f1b96-0570-4f69-a62e-940496a1b239-kube-api-access-7z42h\") pod \"llmisvc-controller-manager-777c6bd7d5-64zpt\" (UID: \"9b8f1b96-0570-4f69-a62e-940496a1b239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" Jul 07 14:38:45.866003 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.865974 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4fsz\" (UniqueName: \"kubernetes.io/projected/130e44c8-ac80-4545-a965-67b82988c0a8-kube-api-access-q4fsz\") pod \"kserve-controller-manager-d6f4b65b8-mx9js\" (UID: \"130e44c8-ac80-4545-a965-67b82988c0a8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" Jul 07 14:38:45.971300 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.971269 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" Jul 07 14:38:45.977278 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:45.977260 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" Jul 07 14:38:46.021425 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:46.021025 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:46.109636 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:46.109569 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mx9js"] Jul 07 14:38:46.113522 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:38:46.113486 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod130e44c8_ac80_4545_a965_67b82988c0a8.slice/crio-fea11c43a17d71b08ffdb4ab5c6014bfd35ce094774f34d7bc366b5db9ea01bb WatchSource:0}: Error finding container fea11c43a17d71b08ffdb4ab5c6014bfd35ce094774f34d7bc366b5db9ea01bb: Status 404 returned error can't find the container with id fea11c43a17d71b08ffdb4ab5c6014bfd35ce094774f34d7bc366b5db9ea01bb Jul 07 14:38:46.136106 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:46.136083 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt"] Jul 07 14:38:46.138885 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:38:46.138863 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9b8f1b96_0570_4f69_a62e_940496a1b239.slice/crio-0cf3fc197aa3471223bef788964c1ad59fc00a49e075bd276fe690984b4fcd65 WatchSource:0}: Error finding container 0cf3fc197aa3471223bef788964c1ad59fc00a49e075bd276fe690984b4fcd65: Status 404 returned error can't find the container with id 0cf3fc197aa3471223bef788964c1ad59fc00a49e075bd276fe690984b4fcd65 Jul 07 14:38:46.713658 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:46.713602 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" event={"ID":"9b8f1b96-0570-4f69-a62e-940496a1b239","Type":"ContainerStarted","Data":"0cf3fc197aa3471223bef788964c1ad59fc00a49e075bd276fe690984b4fcd65"} Jul 07 14:38:46.715123 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:46.715074 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" event={"ID":"130e44c8-ac80-4545-a965-67b82988c0a8","Type":"ContainerStarted","Data":"fea11c43a17d71b08ffdb4ab5c6014bfd35ce094774f34d7bc366b5db9ea01bb"} Jul 07 14:38:46.906223 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:46.906189 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:46.996499 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:46.996419 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:47.166412 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.166381 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:47.463430 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.463400 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-spw6v"] Jul 07 14:38:47.463619 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.463598 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" podUID="f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" containerName="discovery" containerID="cri-o://35041333ab3abe6b8e16365d64da126120496ec7ce791705e790160f8a5e5540" gracePeriod=30 Jul 07 14:38:47.723768 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.723683 2581 generic.go:356] "Generic (PLEG): container finished" podID="f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" containerID="35041333ab3abe6b8e16365d64da126120496ec7ce791705e790160f8a5e5540" exitCode=0 Jul 07 14:38:47.723768 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.723729 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" event={"ID":"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4","Type":"ContainerDied","Data":"35041333ab3abe6b8e16365d64da126120496ec7ce791705e790160f8a5e5540"} Jul 07 14:38:47.844876 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.844849 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:47.964222 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.964186 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds95p\" (UniqueName: \"kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-kube-api-access-ds95p\") pod \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " Jul 07 14:38:47.964222 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.964231 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-token\") pod \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " Jul 07 14:38:47.964901 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.964289 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-cacerts\") pod \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " Jul 07 14:38:47.964901 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.964307 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-dns-cert\") pod \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " Jul 07 14:38:47.964901 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.964357 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-ca-configmap\") pod \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " Jul 07 14:38:47.964901 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.964387 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-kubeconfig\") pod \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " Jul 07 14:38:47.964901 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.964418 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-local-certs\") pod \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\" (UID: \"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4\") " Jul 07 14:38:47.965175 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.964938 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" (UID: "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:38:47.967442 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.967405 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-local-certs" (OuterVolumeSpecName: "local-certs") pod "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" (UID: "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:38:47.967645 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.967613 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-kube-api-access-ds95p" (OuterVolumeSpecName: "kube-api-access-ds95p") pod "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" (UID: "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4"). InnerVolumeSpecName "kube-api-access-ds95p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:38:47.967735 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.967700 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" (UID: "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:38:47.968066 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.968044 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" (UID: "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:38:47.969309 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.969282 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" (UID: "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:38:47.969435 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:47.969389 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-cacerts" (OuterVolumeSpecName: "cacerts") pod "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" (UID: "f6b66eb6-85c0-4e19-941f-3d84bae9a2c4"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:38:48.065476 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.065433 2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-kubeconfig\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:38:48.065627 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.065480 2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-local-certs\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:38:48.065627 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.065499 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds95p\" (UniqueName: \"kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-kube-api-access-ds95p\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:38:48.065627 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.065513 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-token\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:38:48.065627 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.065526 2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-cacerts\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:38:48.065627 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.065538 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-dns-cert\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:38:48.065627 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.065549 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4-istio-csr-ca-configmap\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:38:48.731218 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.731189 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" Jul 07 14:38:48.731646 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.731194 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-spw6v" event={"ID":"f6b66eb6-85c0-4e19-941f-3d84bae9a2c4","Type":"ContainerDied","Data":"0ff26e26bac14d33dc9bd84185548cf123085268d84fd3389ddf8172cde34212"} Jul 07 14:38:48.731646 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.731333 2581 scope.go:117] "RemoveContainer" containerID="35041333ab3abe6b8e16365d64da126120496ec7ce791705e790160f8a5e5540" Jul 07 14:38:48.761151 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.761127 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-spw6v"] Jul 07 14:38:48.766104 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:48.766085 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-spw6v"] Jul 07 14:38:49.246155 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:49.246086 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" path="/var/lib/kubelet/pods/f6b66eb6-85c0-4e19-941f-3d84bae9a2c4/volumes" Jul 07 14:38:49.737956 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:49.737913 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" event={"ID":"9b8f1b96-0570-4f69-a62e-940496a1b239","Type":"ContainerStarted","Data":"26e305ce6a8e11483437327f83e12169a928b8ea3b8ef9e53de4a0e5840b4a4b"} Jul 07 14:38:49.738386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:49.738056 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" Jul 07 14:38:49.739643 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:49.739614 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" event={"ID":"130e44c8-ac80-4545-a965-67b82988c0a8","Type":"ContainerStarted","Data":"f2121fb804b662763fb00fdce386908a14cdc3f7abcde92d5ffcd6e0095336cf"} Jul 07 14:38:49.739761 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:49.739754 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" Jul 07 14:38:49.755579 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:49.755536 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" podStartSLOduration=1.870936269 podStartE2EDuration="4.755524451s" totalImagesPullingTime="2.884588182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:38:49.754547085 +0000 UTC m=+335.029630510" watchObservedRunningTime="2026-07-07 14:38:49.755524451 +0000 UTC m=+335.030607875" Jul 07 14:38:49.773686 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:49.773642 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" podStartSLOduration=1.910828596 podStartE2EDuration="4.77362988s" totalImagesPullingTime="2.862801284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:38:49.773379918 +0000 UTC m=+335.048463344" watchObservedRunningTime="2026-07-07 14:38:49.77362988 +0000 UTC m=+335.048713304" Jul 07 14:38:50.685194 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:38:50.685165 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6hsjf" Jul 07 14:39:20.746282 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:20.746250 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-64zpt" Jul 07 14:39:20.749254 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:20.749235 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx9js" Jul 07 14:39:34.601233 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.601191 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:39:34.694971 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.694941 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:39:34.733275 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.733240 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vszdm"] Jul 07 14:39:34.733649 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.733633 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" containerName="discovery" Jul 07 14:39:34.733743 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.733653 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" containerName="discovery" Jul 07 14:39:34.733795 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.733747 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6b66eb6-85c0-4e19-941f-3d84bae9a2c4" containerName="discovery" Jul 07 14:39:34.741015 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.740991 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vszdm" Jul 07 14:39:34.743876 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.743849 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 14:39:34.744141 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.744121 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-q5hxv\"" Jul 07 14:39:34.744533 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.744351 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 14:39:34.745619 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.745596 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vszdm"] Jul 07 14:39:34.781045 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.781008 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:39:34.851552 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.851471 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2z7r\" (UniqueName: \"kubernetes.io/projected/1b4f7726-0ed7-4c7e-b386-1d2802280a6b-kube-api-access-j2z7r\") pod \"kuadrant-operator-catalog-vszdm\" (UID: \"1b4f7726-0ed7-4c7e-b386-1d2802280a6b\") " pod="kuadrant-system/kuadrant-operator-catalog-vszdm" Jul 07 14:39:34.952737 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.952702 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2z7r\" (UniqueName: \"kubernetes.io/projected/1b4f7726-0ed7-4c7e-b386-1d2802280a6b-kube-api-access-j2z7r\") pod \"kuadrant-operator-catalog-vszdm\" (UID: \"1b4f7726-0ed7-4c7e-b386-1d2802280a6b\") " pod="kuadrant-system/kuadrant-operator-catalog-vszdm" Jul 07 14:39:34.961070 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:34.961038 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2z7r\" (UniqueName: \"kubernetes.io/projected/1b4f7726-0ed7-4c7e-b386-1d2802280a6b-kube-api-access-j2z7r\") pod \"kuadrant-operator-catalog-vszdm\" (UID: \"1b4f7726-0ed7-4c7e-b386-1d2802280a6b\") " pod="kuadrant-system/kuadrant-operator-catalog-vszdm" Jul 07 14:39:35.052331 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.052288 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vszdm" Jul 07 14:39:35.096416 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.096385 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vszdm"] Jul 07 14:39:35.182029 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.182002 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vszdm"] Jul 07 14:39:35.184470 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:39:35.184447 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4f7726_0ed7_4c7e_b386_1d2802280a6b.slice/crio-123a701eb982ce4a9bb3b8e2d0b2de0de7fbd995360aa43ba44cfbbc4f492dc2 WatchSource:0}: Error finding container 123a701eb982ce4a9bb3b8e2d0b2de0de7fbd995360aa43ba44cfbbc4f492dc2: Status 404 returned error can't find the container with id 123a701eb982ce4a9bb3b8e2d0b2de0de7fbd995360aa43ba44cfbbc4f492dc2 Jul 07 14:39:35.306999 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.306961 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6jlfm"] Jul 07 14:39:35.310556 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.310536 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" Jul 07 14:39:35.317445 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.317421 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6jlfm"] Jul 07 14:39:35.356584 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.356551 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c46kv\" (UniqueName: \"kubernetes.io/projected/7fbfcd8e-5fbc-45c4-80bb-d08390e1108d-kube-api-access-c46kv\") pod \"kuadrant-operator-catalog-6jlfm\" (UID: \"7fbfcd8e-5fbc-45c4-80bb-d08390e1108d\") " pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" Jul 07 14:39:35.457931 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.457845 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c46kv\" (UniqueName: \"kubernetes.io/projected/7fbfcd8e-5fbc-45c4-80bb-d08390e1108d-kube-api-access-c46kv\") pod \"kuadrant-operator-catalog-6jlfm\" (UID: \"7fbfcd8e-5fbc-45c4-80bb-d08390e1108d\") " pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" Jul 07 14:39:35.465605 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.465571 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c46kv\" (UniqueName: \"kubernetes.io/projected/7fbfcd8e-5fbc-45c4-80bb-d08390e1108d-kube-api-access-c46kv\") pod \"kuadrant-operator-catalog-6jlfm\" (UID: \"7fbfcd8e-5fbc-45c4-80bb-d08390e1108d\") " pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" Jul 07 14:39:35.620923 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.620891 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" Jul 07 14:39:35.744697 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.744616 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6jlfm"] Jul 07 14:39:35.748369 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:39:35.748346 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fbfcd8e_5fbc_45c4_80bb_d08390e1108d.slice/crio-e11f2fc4377bc4f5be818353ee1f2263209b3b7f2a8a50977fc6e40b16ae2e90 WatchSource:0}: Error finding container e11f2fc4377bc4f5be818353ee1f2263209b3b7f2a8a50977fc6e40b16ae2e90: Status 404 returned error can't find the container with id e11f2fc4377bc4f5be818353ee1f2263209b3b7f2a8a50977fc6e40b16ae2e90 Jul 07 14:39:35.947325 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.947287 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vszdm" event={"ID":"1b4f7726-0ed7-4c7e-b386-1d2802280a6b","Type":"ContainerStarted","Data":"123a701eb982ce4a9bb3b8e2d0b2de0de7fbd995360aa43ba44cfbbc4f492dc2"} Jul 07 14:39:35.948628 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:35.948604 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" event={"ID":"7fbfcd8e-5fbc-45c4-80bb-d08390e1108d","Type":"ContainerStarted","Data":"e11f2fc4377bc4f5be818353ee1f2263209b3b7f2a8a50977fc6e40b16ae2e90"} Jul 07 14:39:37.960032 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:37.959994 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vszdm" event={"ID":"1b4f7726-0ed7-4c7e-b386-1d2802280a6b","Type":"ContainerStarted","Data":"14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3"} Jul 07 14:39:37.960548 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:37.960039 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vszdm" podUID="1b4f7726-0ed7-4c7e-b386-1d2802280a6b" containerName="registry-server" containerID="cri-o://14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3" gracePeriod=2 Jul 07 14:39:37.961659 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:37.961636 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" event={"ID":"7fbfcd8e-5fbc-45c4-80bb-d08390e1108d","Type":"ContainerStarted","Data":"5ca351d3d21699b4a79ba377111d266b9177cf572d09d5a70796a1ba2fcf4624"} Jul 07 14:39:37.977018 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:37.976973 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vszdm" podStartSLOduration=1.814401401 podStartE2EDuration="3.976957626s" totalImagesPullingTime="2.162556225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:39:37.975611089 +0000 UTC m=+383.250694517" watchObservedRunningTime="2026-07-07 14:39:37.976957626 +0000 UTC m=+383.252041051" Jul 07 14:39:37.992399 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:37.992357 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" podStartSLOduration=1.393410407 podStartE2EDuration="2.992342912s" totalImagesPullingTime="1.598932505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:39:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:39:37.991011418 +0000 UTC m=+383.266094867" watchObservedRunningTime="2026-07-07 14:39:37.992342912 +0000 UTC m=+383.267426338" Jul 07 14:39:38.202088 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.202058 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vszdm" Jul 07 14:39:38.284479 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.284388 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2z7r\" (UniqueName: \"kubernetes.io/projected/1b4f7726-0ed7-4c7e-b386-1d2802280a6b-kube-api-access-j2z7r\") pod \"1b4f7726-0ed7-4c7e-b386-1d2802280a6b\" (UID: \"1b4f7726-0ed7-4c7e-b386-1d2802280a6b\") " Jul 07 14:39:38.286596 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.286564 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b4f7726-0ed7-4c7e-b386-1d2802280a6b-kube-api-access-j2z7r" (OuterVolumeSpecName: "kube-api-access-j2z7r") pod "1b4f7726-0ed7-4c7e-b386-1d2802280a6b" (UID: "1b4f7726-0ed7-4c7e-b386-1d2802280a6b"). InnerVolumeSpecName "kube-api-access-j2z7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:39:38.385129 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.385099 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2z7r\" (UniqueName: \"kubernetes.io/projected/1b4f7726-0ed7-4c7e-b386-1d2802280a6b-kube-api-access-j2z7r\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:39:38.967074 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.967041 2581 generic.go:356] "Generic (PLEG): container finished" podID="1b4f7726-0ed7-4c7e-b386-1d2802280a6b" containerID="14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3" exitCode=0 Jul 07 14:39:38.967528 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.967087 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vszdm" Jul 07 14:39:38.967528 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.967133 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vszdm" event={"ID":"1b4f7726-0ed7-4c7e-b386-1d2802280a6b","Type":"ContainerDied","Data":"14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3"} Jul 07 14:39:38.967528 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.967189 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vszdm" event={"ID":"1b4f7726-0ed7-4c7e-b386-1d2802280a6b","Type":"ContainerDied","Data":"123a701eb982ce4a9bb3b8e2d0b2de0de7fbd995360aa43ba44cfbbc4f492dc2"} Jul 07 14:39:38.967528 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.967219 2581 scope.go:117] "RemoveContainer" containerID="14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3" Jul 07 14:39:38.976386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.976365 2581 scope.go:117] "RemoveContainer" containerID="14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3" Jul 07 14:39:38.976641 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:39:38.976624 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3\": container with ID starting with 14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3 not found: ID does not exist" containerID="14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3" Jul 07 14:39:38.976699 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.976650 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3"} err="failed to get container status \"14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3\": rpc error: code = NotFound desc = could not find container \"14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3\": container with ID starting with 14e330e4368900cd04285eecd141907696f53a5f53d491bd85e44e91283b00f3 not found: ID does not exist" Jul 07 14:39:38.991561 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.991526 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vszdm"] Jul 07 14:39:38.996615 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:38.996589 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vszdm"] Jul 07 14:39:39.247634 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:39.247539 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b4f7726-0ed7-4c7e-b386-1d2802280a6b" path="/var/lib/kubelet/pods/1b4f7726-0ed7-4c7e-b386-1d2802280a6b/volumes" Jul 07 14:39:45.621446 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:45.621346 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" Jul 07 14:39:45.622020 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:45.621493 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" Jul 07 14:39:45.644242 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:45.644214 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" Jul 07 14:39:46.021765 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:39:46.021688 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6jlfm" Jul 07 14:40:03.399524 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:03.399495 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:06.017854 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.017807 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c"] Jul 07 14:40:06.018250 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.018193 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b4f7726-0ed7-4c7e-b386-1d2802280a6b" containerName="registry-server" Jul 07 14:40:06.018250 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.018206 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b4f7726-0ed7-4c7e-b386-1d2802280a6b" containerName="registry-server" Jul 07 14:40:06.018361 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.018261 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b4f7726-0ed7-4c7e-b386-1d2802280a6b" containerName="registry-server" Jul 07 14:40:06.023041 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.023022 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c" Jul 07 14:40:06.025855 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.025835 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 07 14:40:06.025952 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.025898 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wsd6p\"" Jul 07 14:40:06.033168 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.033141 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c"] Jul 07 14:40:06.119948 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.119912 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb8bj\" (UniqueName: \"kubernetes.io/projected/d084e5a5-f7a2-436e-9dbb-8d8c9d430284-kube-api-access-mb8bj\") pod \"dns-operator-controller-manager-864dd78c4c-mh72c\" (UID: \"d084e5a5-f7a2-436e-9dbb-8d8c9d430284\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c" Jul 07 14:40:06.220890 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.220849 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb8bj\" (UniqueName: \"kubernetes.io/projected/d084e5a5-f7a2-436e-9dbb-8d8c9d430284-kube-api-access-mb8bj\") pod \"dns-operator-controller-manager-864dd78c4c-mh72c\" (UID: \"d084e5a5-f7a2-436e-9dbb-8d8c9d430284\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c" Jul 07 14:40:06.234451 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.234424 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb8bj\" (UniqueName: \"kubernetes.io/projected/d084e5a5-f7a2-436e-9dbb-8d8c9d430284-kube-api-access-mb8bj\") pod \"dns-operator-controller-manager-864dd78c4c-mh72c\" (UID: \"d084e5a5-f7a2-436e-9dbb-8d8c9d430284\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c" Jul 07 14:40:06.333511 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.333422 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c" Jul 07 14:40:06.471352 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.471330 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c"] Jul 07 14:40:06.473664 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:40:06.473630 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd084e5a5_f7a2_436e_9dbb_8d8c9d430284.slice/crio-250b196d52bc3d401751024cc73eafd8dce97c0844ac8d2be237bca550159819 WatchSource:0}: Error finding container 250b196d52bc3d401751024cc73eafd8dce97c0844ac8d2be237bca550159819: Status 404 returned error can't find the container with id 250b196d52bc3d401751024cc73eafd8dce97c0844ac8d2be237bca550159819 Jul 07 14:40:06.960388 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:06.960325 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:07.095854 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:07.095793 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c" event={"ID":"d084e5a5-f7a2-436e-9dbb-8d8c9d430284","Type":"ContainerStarted","Data":"250b196d52bc3d401751024cc73eafd8dce97c0844ac8d2be237bca550159819"} Jul 07 14:40:09.107422 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:09.107385 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c" event={"ID":"d084e5a5-f7a2-436e-9dbb-8d8c9d430284","Type":"ContainerStarted","Data":"b8726f3c45ed996ffc74872f3ffab4a1163032cf6a3f65ffd317cdf01d8717c3"} Jul 07 14:40:09.107845 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:09.107466 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c" Jul 07 14:40:09.128614 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:09.128562 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c" podStartSLOduration=2.209072538 podStartE2EDuration="4.12854434s" totalImagesPullingTime="1.919471802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:40:09.126539066 +0000 UTC m=+414.401622491" watchObservedRunningTime="2026-07-07 14:40:09.12854434 +0000 UTC m=+414.403627835" Jul 07 14:40:10.760049 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:10.759994 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:12.335094 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.335057 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z"] Jul 07 14:40:12.339276 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.339258 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:12.344190 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.344170 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n556v\"" Jul 07 14:40:12.349730 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.349708 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z"] Jul 07 14:40:12.364386 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.364354 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24b03292-44ff-437f-8cf6-cdd3476b5050-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" (UID: \"24b03292-44ff-437f-8cf6-cdd3476b5050\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:12.364490 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.364457 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khrsk\" (UniqueName: \"kubernetes.io/projected/24b03292-44ff-437f-8cf6-cdd3476b5050-kube-api-access-khrsk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" (UID: \"24b03292-44ff-437f-8cf6-cdd3476b5050\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:12.454972 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.454938 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:12.465143 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.465111 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24b03292-44ff-437f-8cf6-cdd3476b5050-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" (UID: \"24b03292-44ff-437f-8cf6-cdd3476b5050\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:12.465292 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.465162 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khrsk\" (UniqueName: \"kubernetes.io/projected/24b03292-44ff-437f-8cf6-cdd3476b5050-kube-api-access-khrsk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" (UID: \"24b03292-44ff-437f-8cf6-cdd3476b5050\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:12.465522 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.465504 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24b03292-44ff-437f-8cf6-cdd3476b5050-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" (UID: \"24b03292-44ff-437f-8cf6-cdd3476b5050\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:12.478219 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.478191 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khrsk\" (UniqueName: \"kubernetes.io/projected/24b03292-44ff-437f-8cf6-cdd3476b5050-kube-api-access-khrsk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" (UID: \"24b03292-44ff-437f-8cf6-cdd3476b5050\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:12.649765 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.649670 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:12.804381 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.804295 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z"] Jul 07 14:40:12.808452 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:40:12.808412 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b03292_44ff_437f_8cf6_cdd3476b5050.slice/crio-823f60a69d419b0ef6aa667c45562574a8d5d0dbac03a9ef55e1409214f33924 WatchSource:0}: Error finding container 823f60a69d419b0ef6aa667c45562574a8d5d0dbac03a9ef55e1409214f33924: Status 404 returned error can't find the container with id 823f60a69d419b0ef6aa667c45562574a8d5d0dbac03a9ef55e1409214f33924 Jul 07 14:40:12.975627 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:12.975537 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:13.129942 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:13.129904 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" event={"ID":"24b03292-44ff-437f-8cf6-cdd3476b5050","Type":"ContainerStarted","Data":"823f60a69d419b0ef6aa667c45562574a8d5d0dbac03a9ef55e1409214f33924"} Jul 07 14:40:13.755992 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:13.755943 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:16.078605 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:16.078568 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:16.943353 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:16.943317 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq"] Jul 07 14:40:16.947406 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:16.947385 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq" Jul 07 14:40:16.951834 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:16.951792 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-zxf6m\"" Jul 07 14:40:16.958868 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:16.958802 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq"] Jul 07 14:40:16.998292 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:16.998257 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwjvl\" (UniqueName: \"kubernetes.io/projected/4f99bd27-8204-4d2c-9a0e-d3b2576a12b9-kube-api-access-kwjvl\") pod \"authorino-operator-75cc8d5cb7-6xbvq\" (UID: \"4f99bd27-8204-4d2c-9a0e-d3b2576a12b9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq" Jul 07 14:40:17.099395 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:17.099351 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwjvl\" (UniqueName: \"kubernetes.io/projected/4f99bd27-8204-4d2c-9a0e-d3b2576a12b9-kube-api-access-kwjvl\") pod \"authorino-operator-75cc8d5cb7-6xbvq\" (UID: \"4f99bd27-8204-4d2c-9a0e-d3b2576a12b9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq" Jul 07 14:40:17.110227 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:17.110192 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwjvl\" (UniqueName: \"kubernetes.io/projected/4f99bd27-8204-4d2c-9a0e-d3b2576a12b9-kube-api-access-kwjvl\") pod \"authorino-operator-75cc8d5cb7-6xbvq\" (UID: \"4f99bd27-8204-4d2c-9a0e-d3b2576a12b9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq" Jul 07 14:40:17.260775 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:17.260664 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq" Jul 07 14:40:17.840866 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:17.840841 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq"] Jul 07 14:40:17.842690 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:40:17.842654 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f99bd27_8204_4d2c_9a0e_d3b2576a12b9.slice/crio-6ecd1371a5f1314110439ac3618386ba69a5febb6b22425e97bb89f0bcf2cfbb WatchSource:0}: Error finding container 6ecd1371a5f1314110439ac3618386ba69a5febb6b22425e97bb89f0bcf2cfbb: Status 404 returned error can't find the container with id 6ecd1371a5f1314110439ac3618386ba69a5febb6b22425e97bb89f0bcf2cfbb Jul 07 14:40:17.969465 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:17.969383 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:18.156887 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:18.156839 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" event={"ID":"24b03292-44ff-437f-8cf6-cdd3476b5050","Type":"ContainerStarted","Data":"b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94"} Jul 07 14:40:18.157345 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:18.156920 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:18.158076 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:18.158052 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq" event={"ID":"4f99bd27-8204-4d2c-9a0e-d3b2576a12b9","Type":"ContainerStarted","Data":"6ecd1371a5f1314110439ac3618386ba69a5febb6b22425e97bb89f0bcf2cfbb"} Jul 07 14:40:18.177861 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:18.177794 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" podStartSLOduration=1.2424464 podStartE2EDuration="6.177775912s" totalImagesPullingTime="4.935329512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:40:18.175985714 +0000 UTC m=+423.451069141" watchObservedRunningTime="2026-07-07 14:40:18.177775912 +0000 UTC m=+423.452859339" Jul 07 14:40:20.035323 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.035289 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f"] Jul 07 14:40:20.046850 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.046807 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:20.049458 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.049435 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-j5q8t\"" Jul 07 14:40:20.050990 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.050954 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f"] Jul 07 14:40:20.114352 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.114319 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mh72c" Jul 07 14:40:20.128121 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.128088 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmm5j\" (UniqueName: \"kubernetes.io/projected/c4f24562-175f-47e3-a06c-571ee564d78c-kube-api-access-vmm5j\") pod \"limitador-operator-controller-manager-859d7b78bd-sxc9f\" (UID: \"c4f24562-175f-47e3-a06c-571ee564d78c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:20.229078 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.229039 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmm5j\" (UniqueName: \"kubernetes.io/projected/c4f24562-175f-47e3-a06c-571ee564d78c-kube-api-access-vmm5j\") pod \"limitador-operator-controller-manager-859d7b78bd-sxc9f\" (UID: \"c4f24562-175f-47e3-a06c-571ee564d78c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:20.242399 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.242366 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmm5j\" (UniqueName: \"kubernetes.io/projected/c4f24562-175f-47e3-a06c-571ee564d78c-kube-api-access-vmm5j\") pod \"limitador-operator-controller-manager-859d7b78bd-sxc9f\" (UID: \"c4f24562-175f-47e3-a06c-571ee564d78c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:20.358905 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.358863 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:20.518324 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.518301 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f"] Jul 07 14:40:20.520631 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:40:20.520597 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f24562_175f_47e3_a06c_571ee564d78c.slice/crio-871f2e27f1fe0febbb28bafd6437e5fdd8a7ec93d38df2974c3b1ccbe1db7407 WatchSource:0}: Error finding container 871f2e27f1fe0febbb28bafd6437e5fdd8a7ec93d38df2974c3b1ccbe1db7407: Status 404 returned error can't find the container with id 871f2e27f1fe0febbb28bafd6437e5fdd8a7ec93d38df2974c3b1ccbe1db7407 Jul 07 14:40:20.948354 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.948324 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c"] Jul 07 14:40:20.952204 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.952182 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" Jul 07 14:40:20.954950 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.954927 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 14:40:20.955058 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.954931 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t2x5v\"" Jul 07 14:40:20.955058 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.954934 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 14:40:20.957868 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.957843 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:20.963809 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:20.963783 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c"] Jul 07 14:40:21.036567 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.036537 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/45975d7e-613c-4a77-a3a8-a8dc166b75ce-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kmp6c\" (UID: \"45975d7e-613c-4a77-a3a8-a8dc166b75ce\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" Jul 07 14:40:21.036949 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.036574 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc969\" (UniqueName: \"kubernetes.io/projected/45975d7e-613c-4a77-a3a8-a8dc166b75ce-kube-api-access-mc969\") pod \"kuadrant-console-plugin-66b5774985-kmp6c\" (UID: \"45975d7e-613c-4a77-a3a8-a8dc166b75ce\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" Jul 07 14:40:21.036949 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.036686 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/45975d7e-613c-4a77-a3a8-a8dc166b75ce-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kmp6c\" (UID: \"45975d7e-613c-4a77-a3a8-a8dc166b75ce\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" Jul 07 14:40:21.084729 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.084702 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:21.137396 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.137360 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc969\" (UniqueName: \"kubernetes.io/projected/45975d7e-613c-4a77-a3a8-a8dc166b75ce-kube-api-access-mc969\") pod \"kuadrant-console-plugin-66b5774985-kmp6c\" (UID: \"45975d7e-613c-4a77-a3a8-a8dc166b75ce\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" Jul 07 14:40:21.137579 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.137459 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/45975d7e-613c-4a77-a3a8-a8dc166b75ce-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kmp6c\" (UID: \"45975d7e-613c-4a77-a3a8-a8dc166b75ce\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" Jul 07 14:40:21.137579 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.137524 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/45975d7e-613c-4a77-a3a8-a8dc166b75ce-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kmp6c\" (UID: \"45975d7e-613c-4a77-a3a8-a8dc166b75ce\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" Jul 07 14:40:21.138259 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.138227 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/45975d7e-613c-4a77-a3a8-a8dc166b75ce-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kmp6c\" (UID: \"45975d7e-613c-4a77-a3a8-a8dc166b75ce\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" Jul 07 14:40:21.140082 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.140060 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/45975d7e-613c-4a77-a3a8-a8dc166b75ce-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kmp6c\" (UID: \"45975d7e-613c-4a77-a3a8-a8dc166b75ce\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" Jul 07 14:40:21.167393 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.167357 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc969\" (UniqueName: \"kubernetes.io/projected/45975d7e-613c-4a77-a3a8-a8dc166b75ce-kube-api-access-mc969\") pod \"kuadrant-console-plugin-66b5774985-kmp6c\" (UID: \"45975d7e-613c-4a77-a3a8-a8dc166b75ce\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" Jul 07 14:40:21.182495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.182389 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" event={"ID":"c4f24562-175f-47e3-a06c-571ee564d78c","Type":"ContainerStarted","Data":"871f2e27f1fe0febbb28bafd6437e5fdd8a7ec93d38df2974c3b1ccbe1db7407"} Jul 07 14:40:21.184352 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.184326 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq" event={"ID":"4f99bd27-8204-4d2c-9a0e-d3b2576a12b9","Type":"ContainerStarted","Data":"d45b111fdd032b36acdd3ce2c3860d42c24693c172d07fb9c362e59f728006a1"} Jul 07 14:40:21.184471 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.184453 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq" Jul 07 14:40:21.216228 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.216118 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq" podStartSLOduration=2.68850489 podStartE2EDuration="5.216098683s" totalImagesPullingTime="2.527593793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:40:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:40:21.215174735 +0000 UTC m=+426.490258186" watchObservedRunningTime="2026-07-07 14:40:21.216098683 +0000 UTC m=+426.491182108" Jul 07 14:40:21.265123 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.265086 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" Jul 07 14:40:21.467566 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:21.467534 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c"] Jul 07 14:40:21.470565 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:40:21.470535 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45975d7e_613c_4a77_a3a8_a8dc166b75ce.slice/crio-b23b45a9738e372b0af50103f07bd790a8f90afc032a338a1498bd2580829e51 WatchSource:0}: Error finding container b23b45a9738e372b0af50103f07bd790a8f90afc032a338a1498bd2580829e51: Status 404 returned error can't find the container with id b23b45a9738e372b0af50103f07bd790a8f90afc032a338a1498bd2580829e51 Jul 07 14:40:22.190210 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:22.190177 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" event={"ID":"45975d7e-613c-4a77-a3a8-a8dc166b75ce","Type":"ContainerStarted","Data":"b23b45a9738e372b0af50103f07bd790a8f90afc032a338a1498bd2580829e51"} Jul 07 14:40:22.378371 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:22.378337 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:22.476262 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:22.476179 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:23.196439 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:23.196398 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" event={"ID":"c4f24562-175f-47e3-a06c-571ee564d78c","Type":"ContainerStarted","Data":"b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0"} Jul 07 14:40:23.196847 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:23.196527 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:23.219682 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:23.219633 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" podStartSLOduration=1.502094342 podStartE2EDuration="3.219616136s" totalImagesPullingTime="1.717521794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:40:23.217652988 +0000 UTC m=+428.492736466" watchObservedRunningTime="2026-07-07 14:40:23.219616136 +0000 UTC m=+428.494699633" Jul 07 14:40:25.118186 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:25.118149 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:25.213648 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:25.213583 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:26.377199 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:26.377164 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:26.477366 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:26.477334 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:29.165736 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:29.165705 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:31.129953 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.129917 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z"] Jul 07 14:40:31.130398 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.130121 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" podUID="24b03292-44ff-437f-8cf6-cdd3476b5050" containerName="manager" containerID="cri-o://b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94" gracePeriod=2 Jul 07 14:40:31.148768 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.148703 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z"] Jul 07 14:40:31.156210 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:31.156176 2581 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"24b03292-44ff-437f-8cf6-cdd3476b5050\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-7z97z\": pods \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:31.164917 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.164880 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw"] Jul 07 14:40:31.165383 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.165364 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b03292-44ff-437f-8cf6-cdd3476b5050" containerName="manager" Jul 07 14:40:31.165383 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.165385 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b03292-44ff-437f-8cf6-cdd3476b5050" containerName="manager" Jul 07 14:40:31.165537 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.165487 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b03292-44ff-437f-8cf6-cdd3476b5050" containerName="manager" Jul 07 14:40:31.170052 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.170027 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f"] Jul 07 14:40:31.170243 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.170213 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" podUID="c4f24562-175f-47e3-a06c-571ee564d78c" containerName="manager" containerID="cri-o://b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0" gracePeriod=2 Jul 07 14:40:31.170498 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.170479 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:31.172325 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.172304 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:31.187079 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:31.187027 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="24b03292-44ff-437f-8cf6-cdd3476b5050" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:31.189620 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.189058 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f"] Jul 07 14:40:31.192602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.192574 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw"] Jul 07 14:40:31.218944 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.218916 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs"] Jul 07 14:40:31.219408 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.219390 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4f24562-175f-47e3-a06c-571ee564d78c" containerName="manager" Jul 07 14:40:31.219408 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.219408 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4f24562-175f-47e3-a06c-571ee564d78c" containerName="manager" Jul 07 14:40:31.219566 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.219496 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4f24562-175f-47e3-a06c-571ee564d78c" containerName="manager" Jul 07 14:40:31.222585 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.222565 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs" Jul 07 14:40:31.231131 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.231104 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skvjw\" (UID: \"30036cdd-aac1-4aa9-8be7-557ccb1d0e14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:31.231275 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.231247 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c57xf\" (UniqueName: \"kubernetes.io/projected/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-kube-api-access-c57xf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skvjw\" (UID: \"30036cdd-aac1-4aa9-8be7-557ccb1d0e14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:31.234152 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:31.234122 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sxc9f\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="c4f24562-175f-47e3-a06c-571ee564d78c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:31.236343 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:31.236313 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="24b03292-44ff-437f-8cf6-cdd3476b5050" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:31.241372 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:31.240522 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sxc9f\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="c4f24562-175f-47e3-a06c-571ee564d78c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:31.242891 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:31.242864 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="24b03292-44ff-437f-8cf6-cdd3476b5050" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:31.245944 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.245913 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs"] Jul 07 14:40:31.332669 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.332630 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skvjw\" (UID: \"30036cdd-aac1-4aa9-8be7-557ccb1d0e14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:31.332867 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.332702 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db68l\" (UniqueName: \"kubernetes.io/projected/9748b141-7b48-4e02-b30d-522927884290-kube-api-access-db68l\") pod \"limitador-operator-controller-manager-859d7b78bd-v5zfs\" (UID: \"9748b141-7b48-4e02-b30d-522927884290\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs" Jul 07 14:40:31.332867 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.332778 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c57xf\" (UniqueName: \"kubernetes.io/projected/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-kube-api-access-c57xf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skvjw\" (UID: \"30036cdd-aac1-4aa9-8be7-557ccb1d0e14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:31.333086 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.333065 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skvjw\" (UID: \"30036cdd-aac1-4aa9-8be7-557ccb1d0e14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:31.345225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.345195 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c57xf\" (UniqueName: \"kubernetes.io/projected/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-kube-api-access-c57xf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skvjw\" (UID: \"30036cdd-aac1-4aa9-8be7-557ccb1d0e14\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:31.433408 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.433309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db68l\" (UniqueName: \"kubernetes.io/projected/9748b141-7b48-4e02-b30d-522927884290-kube-api-access-db68l\") pod \"limitador-operator-controller-manager-859d7b78bd-v5zfs\" (UID: \"9748b141-7b48-4e02-b30d-522927884290\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs" Jul 07 14:40:31.443089 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.443036 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-db68l\" (UniqueName: \"kubernetes.io/projected/9748b141-7b48-4e02-b30d-522927884290-kube-api-access-db68l\") pod \"limitador-operator-controller-manager-859d7b78bd-v5zfs\" (UID: \"9748b141-7b48-4e02-b30d-522927884290\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs" Jul 07 14:40:31.571712 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.571677 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:31.579011 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:31.578987 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs" Jul 07 14:40:32.036288 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:32.036253 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs"] Jul 07 14:40:32.042792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:32.042772 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" Jul 07 14:40:32.055902 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:32.055874 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs"] Jul 07 14:40:32.141036 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:32.140992 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjrtd\" (UniqueName: \"kubernetes.io/projected/9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc-kube-api-access-bjrtd\") pod \"kuadrant-operator-controller-manager-86878c769f-pjgjs\" (UID: \"9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" Jul 07 14:40:32.141476 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:32.141071 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pjgjs\" (UID: \"9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" Jul 07 14:40:32.192625 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:32.192591 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xbvq" Jul 07 14:40:32.241742 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:32.241705 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjrtd\" (UniqueName: \"kubernetes.io/projected/9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc-kube-api-access-bjrtd\") pod \"kuadrant-operator-controller-manager-86878c769f-pjgjs\" (UID: \"9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" Jul 07 14:40:32.241944 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:32.241770 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pjgjs\" (UID: \"9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" Jul 07 14:40:32.242434 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:32.242403 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pjgjs\" (UID: \"9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" Jul 07 14:40:32.252318 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:32.252290 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjrtd\" (UniqueName: \"kubernetes.io/projected/9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc-kube-api-access-bjrtd\") pod \"kuadrant-operator-controller-manager-86878c769f-pjgjs\" (UID: \"9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" Jul 07 14:40:32.355987 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:32.355941 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" Jul 07 14:40:35.012531 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:35.012497 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:36.374857 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:36.374829 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:36.475666 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:36.475634 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:36.962432 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:36.962401 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:37.031007 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:37.030975 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:37.197420 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:37.197384 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:39.762393 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:39.762362 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:39.862687 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:39.862651 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:43.777961 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:43.777875 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:43.880008 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:43.879950 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:45.652953 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.650853 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs"] Jul 07 14:40:45.656609 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:40:45.656503 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f9f95fe_e9c4_4d19_bd95_4c600fd0d8cc.slice/crio-3fb4e6c534ba2de43805c7bdba3b30acfb4afdb56e96c603a152e995c4ddb4ec WatchSource:0}: Error finding container 3fb4e6c534ba2de43805c7bdba3b30acfb4afdb56e96c603a152e995c4ddb4ec: Status 404 returned error can't find the container with id 3fb4e6c534ba2de43805c7bdba3b30acfb4afdb56e96c603a152e995c4ddb4ec Jul 07 14:40:45.661887 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.661391 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:45.663711 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:45.663679 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sxc9f\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="c4f24562-175f-47e3-a06c-571ee564d78c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:45.664186 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.664168 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:45.666175 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:45.666146 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sxc9f\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="c4f24562-175f-47e3-a06c-571ee564d78c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:45.667961 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:45.667927 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="24b03292-44ff-437f-8cf6-cdd3476b5050" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:45.682454 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.682427 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw"] Jul 07 14:40:45.688790 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:40:45.688755 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30036cdd_aac1_4aa9_8be7_557ccb1d0e14.slice/crio-2008846e014c1e973e3f209c35b42bc8300b2815f3911c97b1aa50f7606a6c36 WatchSource:0}: Error finding container 2008846e014c1e973e3f209c35b42bc8300b2815f3911c97b1aa50f7606a6c36: Status 404 returned error can't find the container with id 2008846e014c1e973e3f209c35b42bc8300b2815f3911c97b1aa50f7606a6c36 Jul 07 14:40:45.706111 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.706050 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs"] Jul 07 14:40:45.709631 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:40:45.709610 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9748b141_7b48_4e02_b30d_522927884290.slice/crio-56f18ebc09d00df6005082f16802154cb44151796589c2302d732c3603d8dc86 WatchSource:0}: Error finding container 56f18ebc09d00df6005082f16802154cb44151796589c2302d732c3603d8dc86: Status 404 returned error can't find the container with id 56f18ebc09d00df6005082f16802154cb44151796589c2302d732c3603d8dc86 Jul 07 14:40:45.772289 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.772264 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmm5j\" (UniqueName: \"kubernetes.io/projected/c4f24562-175f-47e3-a06c-571ee564d78c-kube-api-access-vmm5j\") pod \"c4f24562-175f-47e3-a06c-571ee564d78c\" (UID: \"c4f24562-175f-47e3-a06c-571ee564d78c\") " Jul 07 14:40:45.772535 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.772298 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khrsk\" (UniqueName: \"kubernetes.io/projected/24b03292-44ff-437f-8cf6-cdd3476b5050-kube-api-access-khrsk\") pod \"24b03292-44ff-437f-8cf6-cdd3476b5050\" (UID: \"24b03292-44ff-437f-8cf6-cdd3476b5050\") " Jul 07 14:40:45.772535 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.772415 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24b03292-44ff-437f-8cf6-cdd3476b5050-extensions-socket-volume\") pod \"24b03292-44ff-437f-8cf6-cdd3476b5050\" (UID: \"24b03292-44ff-437f-8cf6-cdd3476b5050\") " Jul 07 14:40:45.772967 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.772938 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24b03292-44ff-437f-8cf6-cdd3476b5050-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "24b03292-44ff-437f-8cf6-cdd3476b5050" (UID: "24b03292-44ff-437f-8cf6-cdd3476b5050"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:40:45.774328 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.774305 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4f24562-175f-47e3-a06c-571ee564d78c-kube-api-access-vmm5j" (OuterVolumeSpecName: "kube-api-access-vmm5j") pod "c4f24562-175f-47e3-a06c-571ee564d78c" (UID: "c4f24562-175f-47e3-a06c-571ee564d78c"). InnerVolumeSpecName "kube-api-access-vmm5j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:40:45.774416 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.774335 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24b03292-44ff-437f-8cf6-cdd3476b5050-kube-api-access-khrsk" (OuterVolumeSpecName: "kube-api-access-khrsk") pod "24b03292-44ff-437f-8cf6-cdd3476b5050" (UID: "24b03292-44ff-437f-8cf6-cdd3476b5050"). InnerVolumeSpecName "kube-api-access-khrsk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:40:45.873754 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.873716 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24b03292-44ff-437f-8cf6-cdd3476b5050-extensions-socket-volume\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:40:45.873754 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.873747 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmm5j\" (UniqueName: \"kubernetes.io/projected/c4f24562-175f-47e3-a06c-571ee564d78c-kube-api-access-vmm5j\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:40:45.873754 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:45.873756 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khrsk\" (UniqueName: \"kubernetes.io/projected/24b03292-44ff-437f-8cf6-cdd3476b5050-kube-api-access-khrsk\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:40:46.328977 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.328943 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" event={"ID":"45975d7e-613c-4a77-a3a8-a8dc166b75ce","Type":"ContainerStarted","Data":"242c9c7889ee765e51214dd731212c0db4ffc93c578c7d76ce6f3064dc9b9d04"} Jul 07 14:40:46.330213 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.330189 2581 generic.go:356] "Generic (PLEG): container finished" podID="24b03292-44ff-437f-8cf6-cdd3476b5050" containerID="b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94" exitCode=0 Jul 07 14:40:46.330319 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.330235 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:46.330319 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.330286 2581 scope.go:117] "RemoveContainer" containerID="b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94" Jul 07 14:40:46.331439 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.331420 2581 generic.go:356] "Generic (PLEG): container finished" podID="c4f24562-175f-47e3-a06c-571ee564d78c" containerID="b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0" exitCode=0 Jul 07 14:40:46.331510 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.331474 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:46.333150 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.333121 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs" event={"ID":"9748b141-7b48-4e02-b30d-522927884290","Type":"ContainerStarted","Data":"7e19f3f469e196d76a7676d0ccddb21faf7186a3a68b29849815361c6380adb9"} Jul 07 14:40:46.333259 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.333162 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs" event={"ID":"9748b141-7b48-4e02-b30d-522927884290","Type":"ContainerStarted","Data":"56f18ebc09d00df6005082f16802154cb44151796589c2302d732c3603d8dc86"} Jul 07 14:40:46.333259 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.333190 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs" Jul 07 14:40:46.335017 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.334997 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" event={"ID":"9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc","Type":"ContainerStarted","Data":"caa0afecc4ddecf50200d36d58ab39700baa6ca49adf09595e3a5fb883505478"} Jul 07 14:40:46.335116 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.335031 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" event={"ID":"9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc","Type":"ContainerStarted","Data":"3fb4e6c534ba2de43805c7bdba3b30acfb4afdb56e96c603a152e995c4ddb4ec"} Jul 07 14:40:46.335202 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.335150 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" Jul 07 14:40:46.336760 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.336723 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" event={"ID":"30036cdd-aac1-4aa9-8be7-557ccb1d0e14","Type":"ContainerStarted","Data":"5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead"} Jul 07 14:40:46.336880 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.336765 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" event={"ID":"30036cdd-aac1-4aa9-8be7-557ccb1d0e14","Type":"ContainerStarted","Data":"2008846e014c1e973e3f209c35b42bc8300b2815f3911c97b1aa50f7606a6c36"} Jul 07 14:40:46.336880 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.336869 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:46.339797 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.339780 2581 scope.go:117] "RemoveContainer" containerID="b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94" Jul 07 14:40:46.340132 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:46.340109 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94\": container with ID starting with b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94 not found: ID does not exist" containerID="b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94" Jul 07 14:40:46.340215 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.340144 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94"} err="failed to get container status \"b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94\": rpc error: code = NotFound desc = could not find container \"b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94\": container with ID starting with b0a9a1f86240f09f7a39401978e178a0ce6a037fd0ad447cf65ad6ed8e59fb94 not found: ID does not exist" Jul 07 14:40:46.340215 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.340166 2581 scope.go:117] "RemoveContainer" containerID="b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0" Jul 07 14:40:46.348310 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.348294 2581 scope.go:117] "RemoveContainer" containerID="b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0" Jul 07 14:40:46.348536 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:46.348522 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0\": container with ID starting with b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0 not found: ID does not exist" containerID="b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0" Jul 07 14:40:46.348601 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.348541 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0"} err="failed to get container status \"b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0\": rpc error: code = NotFound desc = could not find container \"b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0\": container with ID starting with b3a845c5e5325a45fc39a299ba4f6255f5fd4d801cf9c1a4e9856dbe10cb42a0 not found: ID does not exist" Jul 07 14:40:46.364996 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.364949 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kmp6c" podStartSLOduration=2.254523621 podStartE2EDuration="26.364936091s" totalImagesPullingTime="24.11041247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:40:46.363993717 +0000 UTC m=+451.639077136" watchObservedRunningTime="2026-07-07 14:40:46.364936091 +0000 UTC m=+451.640019517" Jul 07 14:40:46.367363 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:46.367340 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sxc9f\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="c4f24562-175f-47e3-a06c-571ee564d78c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:46.371774 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:46.371748 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="24b03292-44ff-437f-8cf6-cdd3476b5050" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:46.401052 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.400996 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs" podStartSLOduration=15.400981187 podStartE2EDuration="15.400981187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:40:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:40:46.400467864 +0000 UTC m=+451.675551467" watchObservedRunningTime="2026-07-07 14:40:46.400981187 +0000 UTC m=+451.676064590" Jul 07 14:40:46.427560 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.427504 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" podStartSLOduration=14.427485885 podStartE2EDuration="14.427485885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:40:46.425452979 +0000 UTC m=+451.700536404" watchObservedRunningTime="2026-07-07 14:40:46.427485885 +0000 UTC m=+451.702569312" Jul 07 14:40:46.448425 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:46.448373 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" podStartSLOduration=15.448357018 podStartE2EDuration="15.448357018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:40:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:40:46.446071046 +0000 UTC m=+451.721154472" watchObservedRunningTime="2026-07-07 14:40:46.448357018 +0000 UTC m=+451.723440443" Jul 07 14:40:46.448590 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:46.448388 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sxc9f\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="c4f24562-175f-47e3-a06c-571ee564d78c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sxc9f" Jul 07 14:40:46.450352 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:46.450331 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7z97z\" is forbidden: User \"system:node:ip-10-0-143-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-146.ec2.internal' and this object" podUID="24b03292-44ff-437f-8cf6-cdd3476b5050" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7z97z" Jul 07 14:40:47.242330 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:47.242297 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24b03292-44ff-437f-8cf6-cdd3476b5050" path="/var/lib/kubelet/pods/24b03292-44ff-437f-8cf6-cdd3476b5050/volumes" Jul 07 14:40:47.242699 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:47.242604 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4f24562-175f-47e3-a06c-571ee564d78c" path="/var/lib/kubelet/pods/c4f24562-175f-47e3-a06c-571ee564d78c/volumes" Jul 07 14:40:47.369573 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:47.369540 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:47.470154 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:47.470120 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:53.852503 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:53.852471 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:55.995016 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:55.994983 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:56.093002 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:56.092975 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:57.347643 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.347615 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:57.348132 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.347687 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5zfs" Jul 07 14:40:57.348132 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.347808 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pjgjs" Jul 07 14:40:57.461830 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.461775 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw"] Jul 07 14:40:57.462045 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.462020 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" podUID="30036cdd-aac1-4aa9-8be7-557ccb1d0e14" containerName="manager" containerID="cri-o://5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead" gracePeriod=10 Jul 07 14:40:57.731161 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.731133 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:57.783532 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.783505 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-extensions-socket-volume\") pod \"30036cdd-aac1-4aa9-8be7-557ccb1d0e14\" (UID: \"30036cdd-aac1-4aa9-8be7-557ccb1d0e14\") " Jul 07 14:40:57.783704 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.783543 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c57xf\" (UniqueName: \"kubernetes.io/projected/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-kube-api-access-c57xf\") pod \"30036cdd-aac1-4aa9-8be7-557ccb1d0e14\" (UID: \"30036cdd-aac1-4aa9-8be7-557ccb1d0e14\") " Jul 07 14:40:57.783954 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.783925 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "30036cdd-aac1-4aa9-8be7-557ccb1d0e14" (UID: "30036cdd-aac1-4aa9-8be7-557ccb1d0e14"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:40:57.785638 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.785589 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-kube-api-access-c57xf" (OuterVolumeSpecName: "kube-api-access-c57xf") pod "30036cdd-aac1-4aa9-8be7-557ccb1d0e14" (UID: "30036cdd-aac1-4aa9-8be7-557ccb1d0e14"). InnerVolumeSpecName "kube-api-access-c57xf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:40:57.884952 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.884865 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-extensions-socket-volume\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:40:57.884952 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:57.884899 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c57xf\" (UniqueName: \"kubernetes.io/projected/30036cdd-aac1-4aa9-8be7-557ccb1d0e14-kube-api-access-c57xf\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:40:58.398328 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:58.398292 2581 generic.go:356] "Generic (PLEG): container finished" podID="30036cdd-aac1-4aa9-8be7-557ccb1d0e14" containerID="5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead" exitCode=0 Jul 07 14:40:58.398749 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:58.398338 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" Jul 07 14:40:58.398749 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:58.398375 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" event={"ID":"30036cdd-aac1-4aa9-8be7-557ccb1d0e14","Type":"ContainerDied","Data":"5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead"} Jul 07 14:40:58.398749 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:58.398426 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw" event={"ID":"30036cdd-aac1-4aa9-8be7-557ccb1d0e14","Type":"ContainerDied","Data":"2008846e014c1e973e3f209c35b42bc8300b2815f3911c97b1aa50f7606a6c36"} Jul 07 14:40:58.398749 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:58.398447 2581 scope.go:117] "RemoveContainer" containerID="5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead" Jul 07 14:40:58.407385 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:58.407369 2581 scope.go:117] "RemoveContainer" containerID="5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead" Jul 07 14:40:58.407641 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:40:58.407623 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead\": container with ID starting with 5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead not found: ID does not exist" containerID="5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead" Jul 07 14:40:58.407686 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:58.407650 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead"} err="failed to get container status \"5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead\": rpc error: code = NotFound desc = could not find container \"5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead\": container with ID starting with 5af4d3bf89ce021392e2297ec83811938a4ac47e7ab9e40576c95422686b3ead not found: ID does not exist" Jul 07 14:40:58.422594 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:58.422571 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw"] Jul 07 14:40:58.433921 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:58.433894 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skvjw"] Jul 07 14:40:58.995425 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:58.995397 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:59.098207 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:59.098178 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:59.242464 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:40:59.242435 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30036cdd-aac1-4aa9-8be7-557ccb1d0e14" path="/var/lib/kubelet/pods/30036cdd-aac1-4aa9-8be7-557ccb1d0e14/volumes" Jul 07 14:41:01.606140 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:01.606108 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:01.734204 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:01.734171 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:05.500500 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:05.500465 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:05.557106 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:05.557077 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:13.942220 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:13.942184 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:14.002921 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:14.002887 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:14.788893 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:14.788862 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:14.852093 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:14.852059 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:15.435276 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:15.435176 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:15.538487 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:15.538456 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:17.536778 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:17.536742 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:18.006547 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.006517 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:18.019938 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.019884 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ntrpc"] Jul 07 14:41:18.020256 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.020242 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30036cdd-aac1-4aa9-8be7-557ccb1d0e14" containerName="manager" Jul 07 14:41:18.020299 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.020257 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="30036cdd-aac1-4aa9-8be7-557ccb1d0e14" containerName="manager" Jul 07 14:41:18.020337 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.020329 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="30036cdd-aac1-4aa9-8be7-557ccb1d0e14" containerName="manager" Jul 07 14:41:18.161578 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.161538 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:18.261190 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.261099 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:18.310944 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.310903 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ntrpc"] Jul 07 14:41:18.310944 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.310950 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ntrpc"] Jul 07 14:41:18.311142 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.311029 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:18.313789 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.313772 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 14:41:18.358711 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.358677 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgc24\" (UniqueName: \"kubernetes.io/projected/77aa561e-d151-4eca-8a45-cd6ff66b5b90-kube-api-access-qgc24\") pod \"limitador-limitador-64c94fd64f-ntrpc\" (UID: \"77aa561e-d151-4eca-8a45-cd6ff66b5b90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:18.358916 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.358750 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77aa561e-d151-4eca-8a45-cd6ff66b5b90-config-file\") pod \"limitador-limitador-64c94fd64f-ntrpc\" (UID: \"77aa561e-d151-4eca-8a45-cd6ff66b5b90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:18.459454 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.459415 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77aa561e-d151-4eca-8a45-cd6ff66b5b90-config-file\") pod \"limitador-limitador-64c94fd64f-ntrpc\" (UID: \"77aa561e-d151-4eca-8a45-cd6ff66b5b90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:18.459646 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.459497 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgc24\" (UniqueName: \"kubernetes.io/projected/77aa561e-d151-4eca-8a45-cd6ff66b5b90-kube-api-access-qgc24\") pod \"limitador-limitador-64c94fd64f-ntrpc\" (UID: \"77aa561e-d151-4eca-8a45-cd6ff66b5b90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:18.460101 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.460080 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77aa561e-d151-4eca-8a45-cd6ff66b5b90-config-file\") pod \"limitador-limitador-64c94fd64f-ntrpc\" (UID: \"77aa561e-d151-4eca-8a45-cd6ff66b5b90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:18.468073 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.468044 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgc24\" (UniqueName: \"kubernetes.io/projected/77aa561e-d151-4eca-8a45-cd6ff66b5b90-kube-api-access-qgc24\") pod \"limitador-limitador-64c94fd64f-ntrpc\" (UID: \"77aa561e-d151-4eca-8a45-cd6ff66b5b90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:18.621145 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.621111 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:18.755545 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.755520 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ntrpc"] Jul 07 14:41:18.758250 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:41:18.758225 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77aa561e_d151_4eca_8a45_cd6ff66b5b90.slice/crio-3022ca8f9a62dd32003dbf429c6b621699619b549350c572252e9d9233bb60bd WatchSource:0}: Error finding container 3022ca8f9a62dd32003dbf429c6b621699619b549350c572252e9d9233bb60bd: Status 404 returned error can't find the container with id 3022ca8f9a62dd32003dbf429c6b621699619b549350c572252e9d9233bb60bd Jul 07 14:41:18.861269 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.861217 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:18.979474 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.979400 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-sxjg8"] Jul 07 14:41:18.983113 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.983092 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-sxjg8" Jul 07 14:41:18.986178 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.986157 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ggb49\"" Jul 07 14:41:18.996535 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:18.996510 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-sxjg8"] Jul 07 14:41:19.013855 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:19.013788 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:19.065405 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:19.065369 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8fjp\" (UniqueName: \"kubernetes.io/projected/8b760bce-d443-4565-b8b1-1271f3f82d61-kube-api-access-s8fjp\") pod \"authorino-58cd955675-sxjg8\" (UID: \"8b760bce-d443-4565-b8b1-1271f3f82d61\") " pod="kuadrant-system/authorino-58cd955675-sxjg8" Jul 07 14:41:19.111430 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:19.111403 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:19.166755 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:19.166724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8fjp\" (UniqueName: \"kubernetes.io/projected/8b760bce-d443-4565-b8b1-1271f3f82d61-kube-api-access-s8fjp\") pod \"authorino-58cd955675-sxjg8\" (UID: \"8b760bce-d443-4565-b8b1-1271f3f82d61\") " pod="kuadrant-system/authorino-58cd955675-sxjg8" Jul 07 14:41:19.176602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:19.176575 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8fjp\" (UniqueName: \"kubernetes.io/projected/8b760bce-d443-4565-b8b1-1271f3f82d61-kube-api-access-s8fjp\") pod \"authorino-58cd955675-sxjg8\" (UID: \"8b760bce-d443-4565-b8b1-1271f3f82d61\") " pod="kuadrant-system/authorino-58cd955675-sxjg8" Jul 07 14:41:19.292991 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:19.292914 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-sxjg8" Jul 07 14:41:19.465901 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:19.465866 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-sxjg8"] Jul 07 14:41:19.476847 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:41:19.472477 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b760bce_d443_4565_b8b1_1271f3f82d61.slice/crio-0563c341d0c74034a4c19ab0f501be494bdd3af31d754ae14f5978cb68826e52 WatchSource:0}: Error finding container 0563c341d0c74034a4c19ab0f501be494bdd3af31d754ae14f5978cb68826e52: Status 404 returned error can't find the container with id 0563c341d0c74034a4c19ab0f501be494bdd3af31d754ae14f5978cb68826e52 Jul 07 14:41:19.505249 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:19.505213 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" event={"ID":"77aa561e-d151-4eca-8a45-cd6ff66b5b90","Type":"ContainerStarted","Data":"3022ca8f9a62dd32003dbf429c6b621699619b549350c572252e9d9233bb60bd"} Jul 07 14:41:19.506733 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:19.506702 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-sxjg8" event={"ID":"8b760bce-d443-4565-b8b1-1271f3f82d61","Type":"ContainerStarted","Data":"0563c341d0c74034a4c19ab0f501be494bdd3af31d754ae14f5978cb68826e52"} Jul 07 14:41:23.112265 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:23.112229 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-sxjg8"] Jul 07 14:41:23.532520 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:23.532414 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" event={"ID":"77aa561e-d151-4eca-8a45-cd6ff66b5b90","Type":"ContainerStarted","Data":"e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927"} Jul 07 14:41:23.532679 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:23.532566 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:23.533969 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:23.533948 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-sxjg8" event={"ID":"8b760bce-d443-4565-b8b1-1271f3f82d61","Type":"ContainerStarted","Data":"1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0"} Jul 07 14:41:23.534076 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:23.533979 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-sxjg8" podUID="8b760bce-d443-4565-b8b1-1271f3f82d61" containerName="authorino" containerID="cri-o://1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0" gracePeriod=30 Jul 07 14:41:23.552193 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:23.552146 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" podStartSLOduration=1.044022255 podStartE2EDuration="5.552128933s" totalImagesPullingTime="4.508106678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:41:23.550589336 +0000 UTC m=+488.825672773" watchObservedRunningTime="2026-07-07 14:41:23.552128933 +0000 UTC m=+488.827212358" Jul 07 14:41:23.566452 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:23.566400 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-sxjg8" podStartSLOduration=1.761582796 podStartE2EDuration="5.566384355s" totalImagesPullingTime="3.804801559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:41:23.565123384 +0000 UTC m=+488.840206809" watchObservedRunningTime="2026-07-07 14:41:23.566384355 +0000 UTC m=+488.841467780" Jul 07 14:41:23.787558 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:23.787496 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-sxjg8" Jul 07 14:41:23.913296 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:23.913260 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8fjp\" (UniqueName: \"kubernetes.io/projected/8b760bce-d443-4565-b8b1-1271f3f82d61-kube-api-access-s8fjp\") pod \"8b760bce-d443-4565-b8b1-1271f3f82d61\" (UID: \"8b760bce-d443-4565-b8b1-1271f3f82d61\") " Jul 07 14:41:23.915325 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:23.915293 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b760bce-d443-4565-b8b1-1271f3f82d61-kube-api-access-s8fjp" (OuterVolumeSpecName: "kube-api-access-s8fjp") pod "8b760bce-d443-4565-b8b1-1271f3f82d61" (UID: "8b760bce-d443-4565-b8b1-1271f3f82d61"). InnerVolumeSpecName "kube-api-access-s8fjp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:41:24.014811 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:24.014774 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8fjp\" (UniqueName: \"kubernetes.io/projected/8b760bce-d443-4565-b8b1-1271f3f82d61-kube-api-access-s8fjp\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:41:24.539714 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:24.539621 2581 generic.go:356] "Generic (PLEG): container finished" podID="8b760bce-d443-4565-b8b1-1271f3f82d61" containerID="1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0" exitCode=0 Jul 07 14:41:24.539714 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:24.539678 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-sxjg8" Jul 07 14:41:24.540294 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:24.539705 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-sxjg8" event={"ID":"8b760bce-d443-4565-b8b1-1271f3f82d61","Type":"ContainerDied","Data":"1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0"} Jul 07 14:41:24.540294 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:24.539753 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-sxjg8" event={"ID":"8b760bce-d443-4565-b8b1-1271f3f82d61","Type":"ContainerDied","Data":"0563c341d0c74034a4c19ab0f501be494bdd3af31d754ae14f5978cb68826e52"} Jul 07 14:41:24.540294 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:24.539773 2581 scope.go:117] "RemoveContainer" containerID="1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0" Jul 07 14:41:24.549226 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:24.549204 2581 scope.go:117] "RemoveContainer" containerID="1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0" Jul 07 14:41:24.549476 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:41:24.549460 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0\": container with ID starting with 1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0 not found: ID does not exist" containerID="1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0" Jul 07 14:41:24.549527 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:24.549484 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0"} err="failed to get container status \"1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0\": rpc error: code = NotFound desc = could not find container \"1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0\": container with ID starting with 1044dda9b848d54c1986bd05c72deb8665f32e52fd051a7b91c5b14e496d4bc0 not found: ID does not exist" Jul 07 14:41:24.562231 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:24.562206 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-sxjg8"] Jul 07 14:41:24.566580 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:24.566553 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-sxjg8"] Jul 07 14:41:25.242866 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:25.242811 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b760bce-d443-4565-b8b1-1271f3f82d61" path="/var/lib/kubelet/pods/8b760bce-d443-4565-b8b1-1271f3f82d61/volumes" Jul 07 14:41:33.231053 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:33.231016 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ntrpc"] Jul 07 14:41:33.231608 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:33.231323 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" podUID="77aa561e-d151-4eca-8a45-cd6ff66b5b90" containerName="limitador" containerID="cri-o://e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927" gracePeriod=30 Jul 07 14:41:33.232219 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:33.232107 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:33.792295 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:33.792234 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:33.906045 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:33.906009 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77aa561e-d151-4eca-8a45-cd6ff66b5b90-config-file\") pod \"77aa561e-d151-4eca-8a45-cd6ff66b5b90\" (UID: \"77aa561e-d151-4eca-8a45-cd6ff66b5b90\") " Jul 07 14:41:33.906301 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:33.906099 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgc24\" (UniqueName: \"kubernetes.io/projected/77aa561e-d151-4eca-8a45-cd6ff66b5b90-kube-api-access-qgc24\") pod \"77aa561e-d151-4eca-8a45-cd6ff66b5b90\" (UID: \"77aa561e-d151-4eca-8a45-cd6ff66b5b90\") " Jul 07 14:41:33.906476 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:33.906449 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77aa561e-d151-4eca-8a45-cd6ff66b5b90-config-file" (OuterVolumeSpecName: "config-file") pod "77aa561e-d151-4eca-8a45-cd6ff66b5b90" (UID: "77aa561e-d151-4eca-8a45-cd6ff66b5b90"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:41:33.908227 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:33.908209 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77aa561e-d151-4eca-8a45-cd6ff66b5b90-kube-api-access-qgc24" (OuterVolumeSpecName: "kube-api-access-qgc24") pod "77aa561e-d151-4eca-8a45-cd6ff66b5b90" (UID: "77aa561e-d151-4eca-8a45-cd6ff66b5b90"). InnerVolumeSpecName "kube-api-access-qgc24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:41:34.007557 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:34.007519 2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77aa561e-d151-4eca-8a45-cd6ff66b5b90-config-file\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:41:34.007557 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:34.007550 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgc24\" (UniqueName: \"kubernetes.io/projected/77aa561e-d151-4eca-8a45-cd6ff66b5b90-kube-api-access-qgc24\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:41:34.595364 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:34.595329 2581 generic.go:356] "Generic (PLEG): container finished" podID="77aa561e-d151-4eca-8a45-cd6ff66b5b90" containerID="e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927" exitCode=0 Jul 07 14:41:34.595737 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:34.595373 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" Jul 07 14:41:34.595737 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:34.595415 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" event={"ID":"77aa561e-d151-4eca-8a45-cd6ff66b5b90","Type":"ContainerDied","Data":"e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927"} Jul 07 14:41:34.595737 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:34.595462 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ntrpc" event={"ID":"77aa561e-d151-4eca-8a45-cd6ff66b5b90","Type":"ContainerDied","Data":"3022ca8f9a62dd32003dbf429c6b621699619b549350c572252e9d9233bb60bd"} Jul 07 14:41:34.595737 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:34.595486 2581 scope.go:117] "RemoveContainer" containerID="e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927" Jul 07 14:41:34.606150 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:34.606131 2581 scope.go:117] "RemoveContainer" containerID="e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927" Jul 07 14:41:34.606443 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:41:34.606424 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927\": container with ID starting with e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927 not found: ID does not exist" containerID="e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927" Jul 07 14:41:34.606509 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:34.606457 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927"} err="failed to get container status \"e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927\": rpc error: code = NotFound desc = could not find container \"e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927\": container with ID starting with e042c844cf35cc125b241876c3c006a32583f8f36dc4d9bdc831e82a9a22b927 not found: ID does not exist" Jul 07 14:41:34.617768 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:34.617743 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ntrpc"] Jul 07 14:41:34.620770 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:34.620745 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ntrpc"] Jul 07 14:41:35.242501 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:35.242466 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77aa561e-d151-4eca-8a45-cd6ff66b5b90" path="/var/lib/kubelet/pods/77aa561e-d151-4eca-8a45-cd6ff66b5b90/volumes" Jul 07 14:41:39.805225 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.805188 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-qlhr8"] Jul 07 14:41:39.805792 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.805773 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77aa561e-d151-4eca-8a45-cd6ff66b5b90" containerName="limitador" Jul 07 14:41:39.805929 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.805796 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="77aa561e-d151-4eca-8a45-cd6ff66b5b90" containerName="limitador" Jul 07 14:41:39.805929 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.805851 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b760bce-d443-4565-b8b1-1271f3f82d61" containerName="authorino" Jul 07 14:41:39.805929 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.805862 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b760bce-d443-4565-b8b1-1271f3f82d61" containerName="authorino" Jul 07 14:41:39.806023 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.805965 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b760bce-d443-4565-b8b1-1271f3f82d61" containerName="authorino" Jul 07 14:41:39.806023 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.805983 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="77aa561e-d151-4eca-8a45-cd6ff66b5b90" containerName="limitador" Jul 07 14:41:39.808233 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.808216 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-qlhr8" Jul 07 14:41:39.811419 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.811396 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 14:41:39.811539 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.811437 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6xmkw\"" Jul 07 14:41:39.819245 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.819223 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-qlhr8"] Jul 07 14:41:39.931674 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.931649 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:39.957776 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.957740 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2a877257-01e2-48bc-8f90-06967c5739a4-data\") pod \"postgres-69fb4594cf-qlhr8\" (UID: \"2a877257-01e2-48bc-8f90-06967c5739a4\") " pod="opendatahub/postgres-69fb4594cf-qlhr8" Jul 07 14:41:39.957934 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:39.957788 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55bjp\" (UniqueName: \"kubernetes.io/projected/2a877257-01e2-48bc-8f90-06967c5739a4-kube-api-access-55bjp\") pod \"postgres-69fb4594cf-qlhr8\" (UID: \"2a877257-01e2-48bc-8f90-06967c5739a4\") " pod="opendatahub/postgres-69fb4594cf-qlhr8" Jul 07 14:41:40.058793 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:40.058713 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2a877257-01e2-48bc-8f90-06967c5739a4-data\") pod \"postgres-69fb4594cf-qlhr8\" (UID: \"2a877257-01e2-48bc-8f90-06967c5739a4\") " pod="opendatahub/postgres-69fb4594cf-qlhr8" Jul 07 14:41:40.058793 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:40.058745 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55bjp\" (UniqueName: \"kubernetes.io/projected/2a877257-01e2-48bc-8f90-06967c5739a4-kube-api-access-55bjp\") pod \"postgres-69fb4594cf-qlhr8\" (UID: \"2a877257-01e2-48bc-8f90-06967c5739a4\") " pod="opendatahub/postgres-69fb4594cf-qlhr8" Jul 07 14:41:40.059160 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:40.059137 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2a877257-01e2-48bc-8f90-06967c5739a4-data\") pod \"postgres-69fb4594cf-qlhr8\" (UID: \"2a877257-01e2-48bc-8f90-06967c5739a4\") " pod="opendatahub/postgres-69fb4594cf-qlhr8" Jul 07 14:41:40.068944 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:40.068920 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55bjp\" (UniqueName: \"kubernetes.io/projected/2a877257-01e2-48bc-8f90-06967c5739a4-kube-api-access-55bjp\") pod \"postgres-69fb4594cf-qlhr8\" (UID: \"2a877257-01e2-48bc-8f90-06967c5739a4\") " pod="opendatahub/postgres-69fb4594cf-qlhr8" Jul 07 14:41:40.119786 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:40.119758 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-qlhr8" Jul 07 14:41:40.253532 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:40.253498 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-qlhr8"] Jul 07 14:41:40.256786 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:41:40.256750 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a877257_01e2_48bc_8f90_06967c5739a4.slice/crio-ca7551ded56ed9fe00ca9742394b5a1e0f257f3791a6e3afa58bec7ecf2d5386 WatchSource:0}: Error finding container ca7551ded56ed9fe00ca9742394b5a1e0f257f3791a6e3afa58bec7ecf2d5386: Status 404 returned error can't find the container with id ca7551ded56ed9fe00ca9742394b5a1e0f257f3791a6e3afa58bec7ecf2d5386 Jul 07 14:41:40.626028 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:40.625993 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-qlhr8" event={"ID":"2a877257-01e2-48bc-8f90-06967c5739a4","Type":"ContainerStarted","Data":"ca7551ded56ed9fe00ca9742394b5a1e0f257f3791a6e3afa58bec7ecf2d5386"} Jul 07 14:41:45.654653 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:45.654615 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-qlhr8" event={"ID":"2a877257-01e2-48bc-8f90-06967c5739a4","Type":"ContainerStarted","Data":"46cc9c9f879a927d6e77a556f4e72ef0ec2823c2380c3d91dfd9384f2142ddba"} Jul 07 14:41:45.655039 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:45.654675 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-qlhr8" Jul 07 14:41:45.672179 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:45.672133 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-qlhr8" podStartSLOduration=1.906586533 podStartE2EDuration="6.672116615s" totalImagesPullingTime="4.765530082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:41:45.670941568 +0000 UTC m=+510.946024996" watchObservedRunningTime="2026-07-07 14:41:45.672116615 +0000 UTC m=+510.947200088" Jul 07 14:41:46.144641 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:46.144608 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:51.689540 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:51.689512 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-qlhr8" Jul 07 14:41:52.084409 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:52.084314 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:52.419537 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:52.419502 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:59.342276 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.342237 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:59.391896 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.391866 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f6c96c569-hwxrn"] Jul 07 14:41:59.539205 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.539174 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:59.632001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.631902 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f6c96c569-hwxrn"] Jul 07 14:41:59.632001 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.631943 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56bd9c557d-h8ltw"] Jul 07 14:41:59.632229 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.632058 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:41:59.635958 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.635923 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 14:41:59.635958 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.635949 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dj5nv\"" Jul 07 14:41:59.636199 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.635922 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 14:41:59.641127 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.641099 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:59.654888 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.654865 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56bd9c557d-h8ltw"] Jul 07 14:41:59.655015 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.654952 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56bd9c557d-h8ltw" Jul 07 14:41:59.723492 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.723459 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3604d738-193c-48cc-9999-e628627c463d-webhook-cert\") pod \"maas-controller-7f6c96c569-hwxrn\" (UID: \"3604d738-193c-48cc-9999-e628627c463d\") " pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:41:59.723641 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.723503 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2a2803b4-1761-4af3-8ce4-a99d3c63d959-webhook-cert\") pod \"maas-controller-56bd9c557d-h8ltw\" (UID: \"2a2803b4-1761-4af3-8ce4-a99d3c63d959\") " pod="opendatahub/maas-controller-56bd9c557d-h8ltw" Jul 07 14:41:59.723641 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.723576 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb9jh\" (UniqueName: \"kubernetes.io/projected/2a2803b4-1761-4af3-8ce4-a99d3c63d959-kube-api-access-wb9jh\") pod \"maas-controller-56bd9c557d-h8ltw\" (UID: \"2a2803b4-1761-4af3-8ce4-a99d3c63d959\") " pod="opendatahub/maas-controller-56bd9c557d-h8ltw" Jul 07 14:41:59.723641 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.723635 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rl8q\" (UniqueName: \"kubernetes.io/projected/3604d738-193c-48cc-9999-e628627c463d-kube-api-access-5rl8q\") pod \"maas-controller-7f6c96c569-hwxrn\" (UID: \"3604d738-193c-48cc-9999-e628627c463d\") " pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:41:59.824652 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.824617 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3604d738-193c-48cc-9999-e628627c463d-webhook-cert\") pod \"maas-controller-7f6c96c569-hwxrn\" (UID: \"3604d738-193c-48cc-9999-e628627c463d\") " pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:41:59.824652 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.824662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2a2803b4-1761-4af3-8ce4-a99d3c63d959-webhook-cert\") pod \"maas-controller-56bd9c557d-h8ltw\" (UID: \"2a2803b4-1761-4af3-8ce4-a99d3c63d959\") " pod="opendatahub/maas-controller-56bd9c557d-h8ltw" Jul 07 14:41:59.824910 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.824786 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb9jh\" (UniqueName: \"kubernetes.io/projected/2a2803b4-1761-4af3-8ce4-a99d3c63d959-kube-api-access-wb9jh\") pod \"maas-controller-56bd9c557d-h8ltw\" (UID: \"2a2803b4-1761-4af3-8ce4-a99d3c63d959\") " pod="opendatahub/maas-controller-56bd9c557d-h8ltw" Jul 07 14:41:59.824910 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.824863 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rl8q\" (UniqueName: \"kubernetes.io/projected/3604d738-193c-48cc-9999-e628627c463d-kube-api-access-5rl8q\") pod \"maas-controller-7f6c96c569-hwxrn\" (UID: \"3604d738-193c-48cc-9999-e628627c463d\") " pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:41:59.827202 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.827172 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3604d738-193c-48cc-9999-e628627c463d-webhook-cert\") pod \"maas-controller-7f6c96c569-hwxrn\" (UID: \"3604d738-193c-48cc-9999-e628627c463d\") " pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:41:59.827321 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.827232 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2a2803b4-1761-4af3-8ce4-a99d3c63d959-webhook-cert\") pod \"maas-controller-56bd9c557d-h8ltw\" (UID: \"2a2803b4-1761-4af3-8ce4-a99d3c63d959\") " pod="opendatahub/maas-controller-56bd9c557d-h8ltw" Jul 07 14:41:59.834273 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.834252 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb9jh\" (UniqueName: \"kubernetes.io/projected/2a2803b4-1761-4af3-8ce4-a99d3c63d959-kube-api-access-wb9jh\") pod \"maas-controller-56bd9c557d-h8ltw\" (UID: \"2a2803b4-1761-4af3-8ce4-a99d3c63d959\") " pod="opendatahub/maas-controller-56bd9c557d-h8ltw" Jul 07 14:41:59.834389 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.834288 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rl8q\" (UniqueName: \"kubernetes.io/projected/3604d738-193c-48cc-9999-e628627c463d-kube-api-access-5rl8q\") pod \"maas-controller-7f6c96c569-hwxrn\" (UID: \"3604d738-193c-48cc-9999-e628627c463d\") " pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:41:59.943510 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.943426 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:41:59.964285 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:41:59.964254 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56bd9c557d-h8ltw" Jul 07 14:42:00.105642 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:00.105610 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f6c96c569-hwxrn"] Jul 07 14:42:00.108468 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:42:00.108431 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3604d738_193c_48cc_9999_e628627c463d.slice/crio-4f9f361a66b74cf7d82a0184bf48d5012e2aa1ad90536c3fd5578b513c6c93b8 WatchSource:0}: Error finding container 4f9f361a66b74cf7d82a0184bf48d5012e2aa1ad90536c3fd5578b513c6c93b8: Status 404 returned error can't find the container with id 4f9f361a66b74cf7d82a0184bf48d5012e2aa1ad90536c3fd5578b513c6c93b8 Jul 07 14:42:00.136248 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:00.136222 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56bd9c557d-h8ltw"] Jul 07 14:42:00.138298 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:42:00.138275 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2803b4_1761_4af3_8ce4_a99d3c63d959.slice/crio-bb2b89327f82fce956040ee03a0bf79fea070a304616f8e383ea6746fdbea77a WatchSource:0}: Error finding container bb2b89327f82fce956040ee03a0bf79fea070a304616f8e383ea6746fdbea77a: Status 404 returned error can't find the container with id bb2b89327f82fce956040ee03a0bf79fea070a304616f8e383ea6746fdbea77a Jul 07 14:42:00.728069 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:00.728031 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" event={"ID":"3604d738-193c-48cc-9999-e628627c463d","Type":"ContainerStarted","Data":"4f9f361a66b74cf7d82a0184bf48d5012e2aa1ad90536c3fd5578b513c6c93b8"} Jul 07 14:42:00.729046 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:00.729029 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56bd9c557d-h8ltw" event={"ID":"2a2803b4-1761-4af3-8ce4-a99d3c63d959","Type":"ContainerStarted","Data":"bb2b89327f82fce956040ee03a0bf79fea070a304616f8e383ea6746fdbea77a"} Jul 07 14:42:02.744678 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:02.744644 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56bd9c557d-h8ltw" event={"ID":"2a2803b4-1761-4af3-8ce4-a99d3c63d959","Type":"ContainerStarted","Data":"299b3af291602927d13d9fea9cff3064f4bc13266da5702a72ebbd9c9f0bd086"} Jul 07 14:42:02.745151 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:02.744722 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56bd9c557d-h8ltw" Jul 07 14:42:02.746483 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:02.746460 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" event={"ID":"3604d738-193c-48cc-9999-e628627c463d","Type":"ContainerStarted","Data":"df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2"} Jul 07 14:42:02.746594 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:02.746507 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:42:02.789596 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:02.789482 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56bd9c557d-h8ltw" podStartSLOduration=1.388332912 podStartE2EDuration="3.789464862s" totalImagesPullingTime="2.40113195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:42:02.786805876 +0000 UTC m=+528.061889323" watchObservedRunningTime="2026-07-07 14:42:02.789464862 +0000 UTC m=+528.064548287" Jul 07 14:42:02.827758 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:02.827705 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" podStartSLOduration=1.40656026 podStartE2EDuration="3.827689187s" totalImagesPullingTime="2.421128927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:42:02.827686028 +0000 UTC m=+528.102769453" watchObservedRunningTime="2026-07-07 14:42:02.827689187 +0000 UTC m=+528.102772614" Jul 07 14:42:02.843777 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:02.843747 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:02.944882 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:02.944846 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:13.760131 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:13.760091 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:42:13.760524 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:13.760180 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56bd9c557d-h8ltw" Jul 07 14:42:13.839954 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:13.839916 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f6c96c569-hwxrn"] Jul 07 14:42:13.840166 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:13.840141 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" podUID="3604d738-193c-48cc-9999-e628627c463d" containerName="manager" containerID="cri-o://df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2" gracePeriod=30 Jul 07 14:42:14.093828 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.093789 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:42:14.151690 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.151655 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rl8q\" (UniqueName: \"kubernetes.io/projected/3604d738-193c-48cc-9999-e628627c463d-kube-api-access-5rl8q\") pod \"3604d738-193c-48cc-9999-e628627c463d\" (UID: \"3604d738-193c-48cc-9999-e628627c463d\") " Jul 07 14:42:14.151992 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.151721 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3604d738-193c-48cc-9999-e628627c463d-webhook-cert\") pod \"3604d738-193c-48cc-9999-e628627c463d\" (UID: \"3604d738-193c-48cc-9999-e628627c463d\") " Jul 07 14:42:14.153864 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.153833 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3604d738-193c-48cc-9999-e628627c463d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3604d738-193c-48cc-9999-e628627c463d" (UID: "3604d738-193c-48cc-9999-e628627c463d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:42:14.153972 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.153875 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3604d738-193c-48cc-9999-e628627c463d-kube-api-access-5rl8q" (OuterVolumeSpecName: "kube-api-access-5rl8q") pod "3604d738-193c-48cc-9999-e628627c463d" (UID: "3604d738-193c-48cc-9999-e628627c463d"). InnerVolumeSpecName "kube-api-access-5rl8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:42:14.252655 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.252611 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rl8q\" (UniqueName: \"kubernetes.io/projected/3604d738-193c-48cc-9999-e628627c463d-kube-api-access-5rl8q\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:42:14.252655 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.252647 2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3604d738-193c-48cc-9999-e628627c463d-webhook-cert\") on node \"ip-10-0-143-146.ec2.internal\" DevicePath \"\"" Jul 07 14:42:14.806379 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.806345 2581 generic.go:356] "Generic (PLEG): container finished" podID="3604d738-193c-48cc-9999-e628627c463d" containerID="df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2" exitCode=0 Jul 07 14:42:14.806811 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.806386 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" event={"ID":"3604d738-193c-48cc-9999-e628627c463d","Type":"ContainerDied","Data":"df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2"} Jul 07 14:42:14.806811 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.806443 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" event={"ID":"3604d738-193c-48cc-9999-e628627c463d","Type":"ContainerDied","Data":"4f9f361a66b74cf7d82a0184bf48d5012e2aa1ad90536c3fd5578b513c6c93b8"} Jul 07 14:42:14.806811 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.806394 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f6c96c569-hwxrn" Jul 07 14:42:14.806811 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.806470 2581 scope.go:117] "RemoveContainer" containerID="df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2" Jul 07 14:42:14.815740 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.815716 2581 scope.go:117] "RemoveContainer" containerID="df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2" Jul 07 14:42:14.816049 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:42:14.816027 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2\": container with ID starting with df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2 not found: ID does not exist" containerID="df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2" Jul 07 14:42:14.816113 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.816060 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2"} err="failed to get container status \"df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2\": rpc error: code = NotFound desc = could not find container \"df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2\": container with ID starting with df7451d02a4f92040db90cdc844ebd394198f959cb75cbd4bf134d1764c2d9e2 not found: ID does not exist" Jul 07 14:42:14.828101 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.828063 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f6c96c569-hwxrn"] Jul 07 14:42:14.832414 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:14.832387 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f6c96c569-hwxrn"] Jul 07 14:42:15.243220 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:15.243189 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3604d738-193c-48cc-9999-e628627c463d" path="/var/lib/kubelet/pods/3604d738-193c-48cc-9999-e628627c463d/volumes" Jul 07 14:42:37.325199 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:37.325162 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:37.844534 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:37.844499 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:37.902014 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:37.901977 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-84b56fdcc7-2vwqc"] Jul 07 14:42:37.902450 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:37.902432 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3604d738-193c-48cc-9999-e628627c463d" containerName="manager" Jul 07 14:42:37.902546 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:37.902453 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3604d738-193c-48cc-9999-e628627c463d" containerName="manager" Jul 07 14:42:37.902602 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:37.902557 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3604d738-193c-48cc-9999-e628627c463d" containerName="manager" Jul 07 14:42:37.908157 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:37.908135 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:37.911359 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:37.911336 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mlqtv\"" Jul 07 14:42:37.911493 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:37.911418 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 07 14:42:37.919085 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:37.919057 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-84b56fdcc7-2vwqc"] Jul 07 14:42:38.074861 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.074802 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a81072e-af15-4b07-b69d-a4d9820b67aa-maas-api-tls\") pod \"maas-api-84b56fdcc7-2vwqc\" (UID: \"9a81072e-af15-4b07-b69d-a4d9820b67aa\") " pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:38.075076 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.074934 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a81072e-af15-4b07-b69d-a4d9820b67aa-openshift-service-ca\") pod \"maas-api-84b56fdcc7-2vwqc\" (UID: \"9a81072e-af15-4b07-b69d-a4d9820b67aa\") " pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:38.075076 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.074965 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6d8l\" (UniqueName: \"kubernetes.io/projected/9a81072e-af15-4b07-b69d-a4d9820b67aa-kube-api-access-d6d8l\") pod \"maas-api-84b56fdcc7-2vwqc\" (UID: \"9a81072e-af15-4b07-b69d-a4d9820b67aa\") " pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:38.176168 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.176084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a81072e-af15-4b07-b69d-a4d9820b67aa-openshift-service-ca\") pod \"maas-api-84b56fdcc7-2vwqc\" (UID: \"9a81072e-af15-4b07-b69d-a4d9820b67aa\") " pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:38.176168 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.176120 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6d8l\" (UniqueName: \"kubernetes.io/projected/9a81072e-af15-4b07-b69d-a4d9820b67aa-kube-api-access-d6d8l\") pod \"maas-api-84b56fdcc7-2vwqc\" (UID: \"9a81072e-af15-4b07-b69d-a4d9820b67aa\") " pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:38.176168 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.176159 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a81072e-af15-4b07-b69d-a4d9820b67aa-maas-api-tls\") pod \"maas-api-84b56fdcc7-2vwqc\" (UID: \"9a81072e-af15-4b07-b69d-a4d9820b67aa\") " pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:38.176456 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:42:38.176290 2581 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 07 14:42:38.176456 ip-10-0-143-146 kubenswrapper[2581]: E0707 14:42:38.176347 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a81072e-af15-4b07-b69d-a4d9820b67aa-maas-api-tls podName:9a81072e-af15-4b07-b69d-a4d9820b67aa nodeName:}" failed. No retries permitted until 2026-07-07 14:42:38.676332274 +0000 UTC m=+563.951415678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9a81072e-af15-4b07-b69d-a4d9820b67aa-maas-api-tls") pod "maas-api-84b56fdcc7-2vwqc" (UID: "9a81072e-af15-4b07-b69d-a4d9820b67aa") : secret "maas-api-serving-cert" not found Jul 07 14:42:38.176761 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.176742 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a81072e-af15-4b07-b69d-a4d9820b67aa-openshift-service-ca\") pod \"maas-api-84b56fdcc7-2vwqc\" (UID: \"9a81072e-af15-4b07-b69d-a4d9820b67aa\") " pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:38.185769 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.185740 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6d8l\" (UniqueName: \"kubernetes.io/projected/9a81072e-af15-4b07-b69d-a4d9820b67aa-kube-api-access-d6d8l\") pod \"maas-api-84b56fdcc7-2vwqc\" (UID: \"9a81072e-af15-4b07-b69d-a4d9820b67aa\") " pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:38.290172 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.290133 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:38.382246 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.382214 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-wch9q"] Jul 07 14:42:38.386257 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.386236 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" Jul 07 14:42:38.386739 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.386717 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:38.388798 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.388781 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 14:42:38.394291 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.394181 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-wch9q"] Jul 07 14:42:38.446408 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.446319 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-vkrl7"] Jul 07 14:42:38.450342 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.450319 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" Jul 07 14:42:38.452910 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.452891 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-gjnmd\"" Jul 07 14:42:38.462073 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.462047 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-vkrl7"] Jul 07 14:42:38.580652 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.580614 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl4hf\" (UniqueName: \"kubernetes.io/projected/4722d3d1-331e-41f8-91f6-d8925d707053-kube-api-access-vl4hf\") pod \"payload-pre-processing-7964457696-wch9q\" (UID: \"4722d3d1-331e-41f8-91f6-d8925d707053\") " pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" Jul 07 14:42:38.580652 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.580653 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a382906a-50f4-408c-ad1b-639740f4e590-plugins-config-volume\") pod \"payload-processing-58f86c7798-vkrl7\" (UID: \"a382906a-50f4-408c-ad1b-639740f4e590\") " pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" Jul 07 14:42:38.580892 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.580802 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7qtk\" (UniqueName: \"kubernetes.io/projected/a382906a-50f4-408c-ad1b-639740f4e590-kube-api-access-w7qtk\") pod \"payload-processing-58f86c7798-vkrl7\" (UID: \"a382906a-50f4-408c-ad1b-639740f4e590\") " pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" Jul 07 14:42:38.580943 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.580902 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4722d3d1-331e-41f8-91f6-d8925d707053-plugins-config-volume\") pod \"payload-pre-processing-7964457696-wch9q\" (UID: \"4722d3d1-331e-41f8-91f6-d8925d707053\") " pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" Jul 07 14:42:38.607637 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.607602 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:38.681939 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.681906 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7qtk\" (UniqueName: \"kubernetes.io/projected/a382906a-50f4-408c-ad1b-639740f4e590-kube-api-access-w7qtk\") pod \"payload-processing-58f86c7798-vkrl7\" (UID: \"a382906a-50f4-408c-ad1b-639740f4e590\") " pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" Jul 07 14:42:38.682118 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.681953 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4722d3d1-331e-41f8-91f6-d8925d707053-plugins-config-volume\") pod \"payload-pre-processing-7964457696-wch9q\" (UID: \"4722d3d1-331e-41f8-91f6-d8925d707053\") " pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" Jul 07 14:42:38.682118 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.681998 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl4hf\" (UniqueName: \"kubernetes.io/projected/4722d3d1-331e-41f8-91f6-d8925d707053-kube-api-access-vl4hf\") pod \"payload-pre-processing-7964457696-wch9q\" (UID: \"4722d3d1-331e-41f8-91f6-d8925d707053\") " pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" Jul 07 14:42:38.682118 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.682016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a382906a-50f4-408c-ad1b-639740f4e590-plugins-config-volume\") pod \"payload-processing-58f86c7798-vkrl7\" (UID: \"a382906a-50f4-408c-ad1b-639740f4e590\") " pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" Jul 07 14:42:38.682118 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.682043 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a81072e-af15-4b07-b69d-a4d9820b67aa-maas-api-tls\") pod \"maas-api-84b56fdcc7-2vwqc\" (UID: \"9a81072e-af15-4b07-b69d-a4d9820b67aa\") " pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:38.682985 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.682959 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a382906a-50f4-408c-ad1b-639740f4e590-plugins-config-volume\") pod \"payload-processing-58f86c7798-vkrl7\" (UID: \"a382906a-50f4-408c-ad1b-639740f4e590\") " pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" Jul 07 14:42:38.683097 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.682969 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4722d3d1-331e-41f8-91f6-d8925d707053-plugins-config-volume\") pod \"payload-pre-processing-7964457696-wch9q\" (UID: \"4722d3d1-331e-41f8-91f6-d8925d707053\") " pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" Jul 07 14:42:38.684465 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.684443 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a81072e-af15-4b07-b69d-a4d9820b67aa-maas-api-tls\") pod \"maas-api-84b56fdcc7-2vwqc\" (UID: \"9a81072e-af15-4b07-b69d-a4d9820b67aa\") " pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:38.691306 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.691276 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7qtk\" (UniqueName: \"kubernetes.io/projected/a382906a-50f4-408c-ad1b-639740f4e590-kube-api-access-w7qtk\") pod \"payload-processing-58f86c7798-vkrl7\" (UID: \"a382906a-50f4-408c-ad1b-639740f4e590\") " pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" Jul 07 14:42:38.691417 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.691397 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl4hf\" (UniqueName: \"kubernetes.io/projected/4722d3d1-331e-41f8-91f6-d8925d707053-kube-api-access-vl4hf\") pod \"payload-pre-processing-7964457696-wch9q\" (UID: \"4722d3d1-331e-41f8-91f6-d8925d707053\") " pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" Jul 07 14:42:38.699294 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.699200 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" Jul 07 14:42:38.762912 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.762874 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" Jul 07 14:42:38.822396 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.822298 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:38.858888 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.858859 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-wch9q"] Jul 07 14:42:38.860873 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:42:38.860784 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4722d3d1_331e_41f8_91f6_d8925d707053.slice/crio-18bda3d03cb0fc5cf5e81e8b8772960857cdf9e50818737c20bb5402a9700126 WatchSource:0}: Error finding container 18bda3d03cb0fc5cf5e81e8b8772960857cdf9e50818737c20bb5402a9700126: Status 404 returned error can't find the container with id 18bda3d03cb0fc5cf5e81e8b8772960857cdf9e50818737c20bb5402a9700126 Jul 07 14:42:38.928248 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.928217 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" event={"ID":"4722d3d1-331e-41f8-91f6-d8925d707053","Type":"ContainerStarted","Data":"18bda3d03cb0fc5cf5e81e8b8772960857cdf9e50818737c20bb5402a9700126"} Jul 07 14:42:38.928331 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.928300 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-vkrl7"] Jul 07 14:42:38.931002 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:42:38.930971 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda382906a_50f4_408c_ad1b_639740f4e590.slice/crio-02bf59cc05aeb99443ee74c2763a0bd33d76ee4ce2dd1339363b856c3a8de0da WatchSource:0}: Error finding container 02bf59cc05aeb99443ee74c2763a0bd33d76ee4ce2dd1339363b856c3a8de0da: Status 404 returned error can't find the container with id 02bf59cc05aeb99443ee74c2763a0bd33d76ee4ce2dd1339363b856c3a8de0da Jul 07 14:42:38.986781 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:38.986755 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-84b56fdcc7-2vwqc"] Jul 07 14:42:38.988876 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:42:38.988852 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a81072e_af15_4b07_b69d_a4d9820b67aa.slice/crio-9ecce21447a3ee8c4cf7f040996ebe9db1e552795457fff4fcbde7acf43f6768 WatchSource:0}: Error finding container 9ecce21447a3ee8c4cf7f040996ebe9db1e552795457fff4fcbde7acf43f6768: Status 404 returned error can't find the container with id 9ecce21447a3ee8c4cf7f040996ebe9db1e552795457fff4fcbde7acf43f6768 Jul 07 14:42:39.938721 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:39.938678 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-84b56fdcc7-2vwqc" event={"ID":"9a81072e-af15-4b07-b69d-a4d9820b67aa","Type":"ContainerStarted","Data":"9ecce21447a3ee8c4cf7f040996ebe9db1e552795457fff4fcbde7acf43f6768"} Jul 07 14:42:39.940838 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:39.940788 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" event={"ID":"a382906a-50f4-408c-ad1b-639740f4e590","Type":"ContainerStarted","Data":"02bf59cc05aeb99443ee74c2763a0bd33d76ee4ce2dd1339363b856c3a8de0da"} Jul 07 14:42:40.947272 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:40.947240 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" event={"ID":"a382906a-50f4-408c-ad1b-639740f4e590","Type":"ContainerStarted","Data":"b32671b72f7a168d04362d28774b21b48841728c96794e045fc4127ce3a5df06"} Jul 07 14:42:40.947683 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:40.947351 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" Jul 07 14:42:40.950318 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:40.950241 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" event={"ID":"4722d3d1-331e-41f8-91f6-d8925d707053","Type":"ContainerStarted","Data":"fd42ce9b64b5c728932a50e89fc75b04ff1dfba049dcb56547873b7029a33da5"} Jul 07 14:42:40.950526 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:40.950501 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" Jul 07 14:42:40.965647 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:40.965592 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" podStartSLOduration=1.5800502939999999 podStartE2EDuration="2.965571048s" totalImagesPullingTime="1.385520754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:42:40.963156478 +0000 UTC m=+566.238239939" watchObservedRunningTime="2026-07-07 14:42:40.965571048 +0000 UTC m=+566.240654476" Jul 07 14:42:40.979469 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:40.979415 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" podStartSLOduration=1.5253839949999999 podStartE2EDuration="2.979397792s" totalImagesPullingTime="1.454013797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:42:40.97785437 +0000 UTC m=+566.252937819" watchObservedRunningTime="2026-07-07 14:42:40.979397792 +0000 UTC m=+566.254481219" Jul 07 14:42:41.956633 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:41.956598 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-84b56fdcc7-2vwqc" event={"ID":"9a81072e-af15-4b07-b69d-a4d9820b67aa","Type":"ContainerStarted","Data":"f75be0dee91a73a68f71c06809652df0d2c1e087b60d65ad4a907b61e0e7921e"} Jul 07 14:42:41.957102 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:41.957077 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:41.976291 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:41.976234 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-84b56fdcc7-2vwqc" podStartSLOduration=3.018758585 podStartE2EDuration="4.976216497s" totalImagesPullingTime="1.957457912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:42:41.974124269 +0000 UTC m=+567.249207695" watchObservedRunningTime="2026-07-07 14:42:41.976216497 +0000 UTC m=+567.251299923" Jul 07 14:42:45.153136 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:45.153107 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:48.974921 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:48.974883 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:42:50.129305 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:50.129269 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:50.226657 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:50.226626 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:51.958414 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:51.958386 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-vkrl7" Jul 07 14:42:51.958887 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:42:51.958695 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-wch9q" Jul 07 14:43:00.670059 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.670024 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt"] Jul 07 14:43:00.674022 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.674001 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.678800 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.678781 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 14:43:00.680093 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.680074 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 14:43:00.680223 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.680102 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8w44s\"" Jul 07 14:43:00.680223 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.680114 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 14:43:00.687615 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.687587 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt"] Jul 07 14:43:00.722007 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.721969 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:43:00.775069 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.775027 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.775069 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.775067 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.775281 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.775108 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rlwx\" (UniqueName: \"kubernetes.io/projected/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-kube-api-access-6rlwx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.775281 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.775143 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.775281 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.775167 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.775281 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.775241 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.876111 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.876073 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.876307 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.876143 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.876307 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.876167 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.876307 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.876211 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rlwx\" (UniqueName: \"kubernetes.io/projected/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-kube-api-access-6rlwx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.876307 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.876262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.876307 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.876298 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.876731 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.876704 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.876876 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.876761 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.876876 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.876801 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.878500 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.878470 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.878660 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.878640 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.885750 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.885728 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rlwx\" (UniqueName: \"kubernetes.io/projected/47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe-kube-api-access-6rlwx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt\" (UID: \"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:00.986526 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:00.986442 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:01.135401 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:01.135371 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt"] Jul 07 14:43:01.137033 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:43:01.136994 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47c4cfbd_d6de_4ac3_b4ae_7b9e9d85ecbe.slice/crio-ae9617c5333508d01db5a2d70125a00dba66d16c1c4048e0f976c4fba2b77245 WatchSource:0}: Error finding container ae9617c5333508d01db5a2d70125a00dba66d16c1c4048e0f976c4fba2b77245: Status 404 returned error can't find the container with id ae9617c5333508d01db5a2d70125a00dba66d16c1c4048e0f976c4fba2b77245 Jul 07 14:43:02.064899 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:02.064862 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" event={"ID":"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe","Type":"ContainerStarted","Data":"ae9617c5333508d01db5a2d70125a00dba66d16c1c4048e0f976c4fba2b77245"} Jul 07 14:43:03.072084 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:03.072041 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" event={"ID":"47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe","Type":"ContainerStarted","Data":"3e638e76275433de9bad3d9ad398cf4c3ffd0a8c3f02fb93f9b39d294facb047"} Jul 07 14:43:03.089977 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:03.089922 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" podStartSLOduration=1.795063654 podStartE2EDuration="3.089905922s" totalImagesPullingTime="1.294842268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:43:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:43:03.08804129 +0000 UTC m=+588.363124739" watchObservedRunningTime="2026-07-07 14:43:03.089905922 +0000 UTC m=+588.364989348" Jul 07 14:43:08.518801 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:08.518767 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:43:10.986894 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:10.986857 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:10.987273 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:10.986922 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:10.999008 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:10.998981 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:11.125155 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:11.125123 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt" Jul 07 14:43:15.167179 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:15.167150 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6rwzn_88bef153-2452-4b8c-84f4-333d8426694d/console-operator/1.log" Jul 07 14:43:15.167618 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:15.167565 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6rwzn_88bef153-2452-4b8c-84f4-333d8426694d/console-operator/1.log" Jul 07 14:43:20.701674 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:20.701639 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:43:24.683686 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.683654 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl"] Jul 07 14:43:24.687044 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.687028 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.689576 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.689556 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 14:43:24.696904 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.696872 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl"] Jul 07 14:43:24.771081 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.771051 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:43:24.793495 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.793452 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.793689 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.793497 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lfbp\" (UniqueName: \"kubernetes.io/projected/7b478dc0-23e6-4d51-982f-12643fce0bd4-kube-api-access-4lfbp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.793689 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.793528 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.793689 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.793550 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b478dc0-23e6-4d51-982f-12643fce0bd4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.793689 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.793583 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.793689 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.793613 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.894202 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.894164 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.894202 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.894209 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.894432 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.894278 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.894432 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.894296 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lfbp\" (UniqueName: \"kubernetes.io/projected/7b478dc0-23e6-4d51-982f-12643fce0bd4-kube-api-access-4lfbp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.894432 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.894319 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.894432 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.894346 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b478dc0-23e6-4d51-982f-12643fce0bd4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.894714 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.894689 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.894714 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.894713 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.894881 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.894758 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.896574 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.896553 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b478dc0-23e6-4d51-982f-12643fce0bd4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.896746 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.896729 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b478dc0-23e6-4d51-982f-12643fce0bd4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.902589 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.902562 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lfbp\" (UniqueName: \"kubernetes.io/projected/7b478dc0-23e6-4d51-982f-12643fce0bd4-kube-api-access-4lfbp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl\" (UID: \"7b478dc0-23e6-4d51-982f-12643fce0bd4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:24.998796 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:24.998711 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:25.143026 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:25.142963 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl"] Jul 07 14:43:25.145515 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:43:25.145491 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b478dc0_23e6_4d51_982f_12643fce0bd4.slice/crio-b1a56d9dd6051772f027c80e1397488877ef7ac358e91eaf917ffc84dd0ada1d WatchSource:0}: Error finding container b1a56d9dd6051772f027c80e1397488877ef7ac358e91eaf917ffc84dd0ada1d: Status 404 returned error can't find the container with id b1a56d9dd6051772f027c80e1397488877ef7ac358e91eaf917ffc84dd0ada1d Jul 07 14:43:25.146998 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:25.146969 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:43:25.183733 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:25.183698 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" event={"ID":"7b478dc0-23e6-4d51-982f-12643fce0bd4","Type":"ContainerStarted","Data":"b1a56d9dd6051772f027c80e1397488877ef7ac358e91eaf917ffc84dd0ada1d"} Jul 07 14:43:26.191740 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:26.191703 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" event={"ID":"7b478dc0-23e6-4d51-982f-12643fce0bd4","Type":"ContainerStarted","Data":"002597a632aa6cf7b57384fa0cc7fccc5e7cfbbe794a87386db9fc793c67b98c"} Jul 07 14:43:26.210273 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:26.210220 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" podStartSLOduration=1.992952598 podStartE2EDuration="2.210204039s" totalImagesPullingTime="217.251441ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:43:26.208209638 +0000 UTC m=+611.483293097" watchObservedRunningTime="2026-07-07 14:43:26.210204039 +0000 UTC m=+611.485287464" Jul 07 14:43:29.083570 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:29.083522 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:43:34.065329 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:34.065297 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:43:34.999066 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:34.999026 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:34.999261 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:34.999210 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:35.011305 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:35.011284 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:43:35.249646 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:43:35.249567 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl" Jul 07 14:44:11.328090 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:11.328057 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:44:11.430767 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:11.430735 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:44:31.961913 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:31.961882 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:44:32.062899 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:32.062862 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:44:49.684896 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.684863 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-74fbdd588f-tlrnt"] Jul 07 14:44:49.688616 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.688594 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74fbdd588f-tlrnt" Jul 07 14:44:49.692352 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.692331 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 14:44:49.692460 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.692331 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ggb49\"" Jul 07 14:44:49.698244 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.698220 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74fbdd588f-tlrnt"] Jul 07 14:44:49.747691 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.747656 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8c848b95-0e36-4fc7-a332-de4ff667546e-tls-cert\") pod \"authorino-74fbdd588f-tlrnt\" (UID: \"8c848b95-0e36-4fc7-a332-de4ff667546e\") " pod="kuadrant-system/authorino-74fbdd588f-tlrnt" Jul 07 14:44:49.747871 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.747709 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wclm\" (UniqueName: \"kubernetes.io/projected/8c848b95-0e36-4fc7-a332-de4ff667546e-kube-api-access-4wclm\") pod \"authorino-74fbdd588f-tlrnt\" (UID: \"8c848b95-0e36-4fc7-a332-de4ff667546e\") " pod="kuadrant-system/authorino-74fbdd588f-tlrnt" Jul 07 14:44:49.849102 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.849063 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8c848b95-0e36-4fc7-a332-de4ff667546e-tls-cert\") pod \"authorino-74fbdd588f-tlrnt\" (UID: \"8c848b95-0e36-4fc7-a332-de4ff667546e\") " pod="kuadrant-system/authorino-74fbdd588f-tlrnt" Jul 07 14:44:49.849275 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.849117 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wclm\" (UniqueName: \"kubernetes.io/projected/8c848b95-0e36-4fc7-a332-de4ff667546e-kube-api-access-4wclm\") pod \"authorino-74fbdd588f-tlrnt\" (UID: \"8c848b95-0e36-4fc7-a332-de4ff667546e\") " pod="kuadrant-system/authorino-74fbdd588f-tlrnt" Jul 07 14:44:49.851581 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.851553 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8c848b95-0e36-4fc7-a332-de4ff667546e-tls-cert\") pod \"authorino-74fbdd588f-tlrnt\" (UID: \"8c848b95-0e36-4fc7-a332-de4ff667546e\") " pod="kuadrant-system/authorino-74fbdd588f-tlrnt" Jul 07 14:44:49.856671 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.856644 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wclm\" (UniqueName: \"kubernetes.io/projected/8c848b95-0e36-4fc7-a332-de4ff667546e-kube-api-access-4wclm\") pod \"authorino-74fbdd588f-tlrnt\" (UID: \"8c848b95-0e36-4fc7-a332-de4ff667546e\") " pod="kuadrant-system/authorino-74fbdd588f-tlrnt" Jul 07 14:44:49.998486 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:49.998385 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74fbdd588f-tlrnt" Jul 07 14:44:50.135397 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:50.135374 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74fbdd588f-tlrnt"] Jul 07 14:44:50.138670 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:44:50.138638 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c848b95_0e36_4fc7_a332_de4ff667546e.slice/crio-53e7707d101bbb470c5644165b51bdef24baadcb73e05c8687db93679851360d WatchSource:0}: Error finding container 53e7707d101bbb470c5644165b51bdef24baadcb73e05c8687db93679851360d: Status 404 returned error can't find the container with id 53e7707d101bbb470c5644165b51bdef24baadcb73e05c8687db93679851360d Jul 07 14:44:50.619857 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:50.619793 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74fbdd588f-tlrnt" event={"ID":"8c848b95-0e36-4fc7-a332-de4ff667546e","Type":"ContainerStarted","Data":"53e7707d101bbb470c5644165b51bdef24baadcb73e05c8687db93679851360d"} Jul 07 14:44:51.628785 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:51.628742 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74fbdd588f-tlrnt" event={"ID":"8c848b95-0e36-4fc7-a332-de4ff667546e","Type":"ContainerStarted","Data":"9f2d767549d0a94b9d348f098b630a80a5ca6c2c7a3bda79c4d50535bdfdbe9e"} Jul 07 14:44:51.648187 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:51.648130 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-74fbdd588f-tlrnt" podStartSLOduration=2.060703078 podStartE2EDuration="2.648111032s" totalImagesPullingTime="587.407954ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:44:51.646585222 +0000 UTC m=+696.921668640" watchObservedRunningTime="2026-07-07 14:44:51.648111032 +0000 UTC m=+696.923194466" Jul 07 14:44:52.277352 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:52.277303 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:44:52.343228 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:52.343196 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:44:53.562638 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:53.562606 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:44:53.650259 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:53.650213 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:44:59.743043 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:59.743013 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:44:59.841945 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:59.841910 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:44:59.942603 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:44:59.942558 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:00.485879 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:00.485844 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:10.254565 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:10.254487 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:10.354771 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:10.354739 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:10.456557 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:10.456525 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:15.555447 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:15.555414 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:15.615924 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:15.615894 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:27.923780 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:27.923735 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:27.981645 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:27.981612 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:36.887682 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:36.887648 2581 generic.go:356] "Generic (PLEG): container finished" podID="9a81072e-af15-4b07-b69d-a4d9820b67aa" containerID="f75be0dee91a73a68f71c06809652df0d2c1e087b60d65ad4a907b61e0e7921e" exitCode=137 Jul 07 14:45:36.888140 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:36.887724 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-84b56fdcc7-2vwqc" event={"ID":"9a81072e-af15-4b07-b69d-a4d9820b67aa","Type":"ContainerDied","Data":"f75be0dee91a73a68f71c06809652df0d2c1e087b60d65ad4a907b61e0e7921e"} Jul 07 14:45:36.888140 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:36.888124 2581 scope.go:117] "RemoveContainer" containerID="f75be0dee91a73a68f71c06809652df0d2c1e087b60d65ad4a907b61e0e7921e" Jul 07 14:45:37.084643 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:37.084597 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74fbdd588f-tlrnt_8c848b95-0e36-4fc7-a332-de4ff667546e/authorino/0.log" Jul 07 14:45:37.894532 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:37.894502 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-84b56fdcc7-2vwqc" event={"ID":"9a81072e-af15-4b07-b69d-a4d9820b67aa","Type":"ContainerStarted","Data":"eee3d5dd38607b88080b0fc456a23d37c58e3f9982d054847314dca3e88a0453"} Jul 07 14:45:37.894926 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:37.894698 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:45:42.332610 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:42.332578 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mx9js_130e44c8-ac80-4545-a965-67b82988c0a8/manager/0.log" Jul 07 14:45:42.463454 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:42.463422 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-64zpt_9b8f1b96-0570-4f69-a62e-940496a1b239/manager/0.log" Jul 07 14:45:42.605986 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:42.605955 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-84b56fdcc7-2vwqc_9a81072e-af15-4b07-b69d-a4d9820b67aa/maas-api/1.log" Jul 07 14:45:42.880702 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:42.880613 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56bd9c557d-h8ltw_2a2803b4-1761-4af3-8ce4-a99d3c63d959/manager/0.log" Jul 07 14:45:43.008328 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:43.008300 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6hsjf_317793be-a149-4f2e-84c8-699683a38292/server/0.log" Jul 07 14:45:43.290147 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:43.290070 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cd69c54fc-gdlmw_6d06cd06-e75a-49ff-8c91-f36581f6efac/manager/0.log" Jul 07 14:45:43.299000 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:43.298973 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cd69c54fc-gdlmw_6d06cd06-e75a-49ff-8c91-f36581f6efac/copy-manifests/0.log" Jul 07 14:45:43.720833 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:43.720784 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-qlhr8_2a877257-01e2-48bc-8f90-06967c5739a4/postgres/0.log" Jul 07 14:45:43.903777 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:43.903745 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-84b56fdcc7-2vwqc" Jul 07 14:45:45.209022 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:45.208980 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74fbdd588f-tlrnt_8c848b95-0e36-4fc7-a332-de4ff667546e/authorino/0.log" Jul 07 14:45:45.343126 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:45.343085 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6xbvq_4f99bd27-8204-4d2c-9a0e-d3b2576a12b9/manager/0.log" Jul 07 14:45:45.469628 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:45.469525 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mh72c_d084e5a5-f7a2-436e-9dbb-8d8c9d430284/manager/0.log" Jul 07 14:45:45.590261 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:45.590236 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kmp6c_45975d7e-613c-4a77-a3a8-a8dc166b75ce/kuadrant-console-plugin/0.log" Jul 07 14:45:45.721286 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:45.721172 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6jlfm_7fbfcd8e-5fbc-45c4-80bb-d08390e1108d/registry-server/0.log" Jul 07 14:45:45.860195 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:45.860170 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pjgjs_9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc/manager/0.log" Jul 07 14:45:46.133404 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:46.133370 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-v5zfs_9748b141-7b48-4e02-b30d-522927884290/manager/0.log" Jul 07 14:45:46.800712 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:46.800682 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54bbbc8986-7sghp_1e2c220f-9325-4a4e-bd79-38ec5bf729c5/kube-auth-proxy/0.log" Jul 07 14:45:47.214063 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:47.214034 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-wch9q_4722d3d1-331e-41f8-91f6-d8925d707053/payload-pre-processing/0.log" Jul 07 14:45:47.351556 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:47.351517 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-vkrl7_a382906a-50f4-408c-ad1b-639740f4e590/payload-processing/0.log" Jul 07 14:45:47.476801 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:47.476718 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67bf4497f9-krpmz_2485c568-9c6b-4bfe-8a8e-9437042d1f9f/router/0.log" Jul 07 14:45:47.866257 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:47.866227 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-zmxpt_47c4cfbd-d6de-4ac3-b4ae-7b9e9d85ecbe/main/0.log" Jul 07 14:45:48.271506 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:48.271418 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbnxxrl_7b478dc0-23e6-4d51-982f-12643fce0bd4/main/0.log" Jul 07 14:45:53.000285 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:53.000247 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:53.082833 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:53.082790 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:55.920695 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:55.920665 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xb5ll_91b21080-c287-47e7-9829-601c1be484f2/global-pull-secret-syncer/0.log" Jul 07 14:45:55.965924 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:55.965894 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fcqkx_4a097e24-d671-4a5a-96c4-c3f50e38d5e3/konnectivity-agent/0.log" Jul 07 14:45:56.074873 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:45:56.074841 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-146.ec2.internal_00c897cdd111a3fcc4bf6f54f3d25813/haproxy/0.log" Jul 07 14:46:01.134750 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:01.134717 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74fbdd588f-tlrnt_8c848b95-0e36-4fc7-a332-de4ff667546e/authorino/0.log" Jul 07 14:46:01.167658 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:01.167625 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6xbvq_4f99bd27-8204-4d2c-9a0e-d3b2576a12b9/manager/0.log" Jul 07 14:46:01.195133 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:01.195105 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mh72c_d084e5a5-f7a2-436e-9dbb-8d8c9d430284/manager/0.log" Jul 07 14:46:01.232193 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:01.232167 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kmp6c_45975d7e-613c-4a77-a3a8-a8dc166b75ce/kuadrant-console-plugin/0.log" Jul 07 14:46:01.266608 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:01.266581 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6jlfm_7fbfcd8e-5fbc-45c4-80bb-d08390e1108d/registry-server/0.log" Jul 07 14:46:01.326267 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:01.326237 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pjgjs_9f9f95fe-e9c4-4d19-bd95-4c600fd0d8cc/manager/0.log" Jul 07 14:46:01.390473 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:01.390390 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-v5zfs_9748b141-7b48-4e02-b30d-522927884290/manager/0.log" Jul 07 14:46:03.434200 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:03.434165 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-4c5x2_ffc93eac-e8ab-406a-96df-22c97fec6622/cluster-monitoring-operator/0.log" Jul 07 14:46:03.627349 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:03.627319 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5krnj_4a9e3c44-79ac-4e07-b853-d58ce0684394/node-exporter/0.log" Jul 07 14:46:03.653176 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:03.653142 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5krnj_4a9e3c44-79ac-4e07-b853-d58ce0684394/kube-rbac-proxy/0.log" Jul 07 14:46:03.677118 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:03.677089 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5krnj_4a9e3c44-79ac-4e07-b853-d58ce0684394/init-textfile/0.log" Jul 07 14:46:05.411845 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.411794 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2"] Jul 07 14:46:05.415683 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.415662 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.418319 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.418301 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b225k\"/\"kube-root-ca.crt\"" Jul 07 14:46:05.418393 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.418377 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b225k\"/\"openshift-service-ca.crt\"" Jul 07 14:46:05.419283 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.419263 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b225k\"/\"default-dockercfg-cg8l2\"" Jul 07 14:46:05.423079 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.423055 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2"] Jul 07 14:46:05.447576 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.447542 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-podres\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.447576 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.447579 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-proc\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.447842 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.447598 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl59m\" (UniqueName: \"kubernetes.io/projected/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-kube-api-access-dl59m\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.447842 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.447617 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-sys\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.447842 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.447644 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-lib-modules\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.548757 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.548727 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-lib-modules\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.548971 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.548846 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-podres\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.548971 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.548873 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-proc\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.548971 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.548890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl59m\" (UniqueName: \"kubernetes.io/projected/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-kube-api-access-dl59m\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.548971 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.548908 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-sys\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.548971 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.548931 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-lib-modules\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.548971 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.548968 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-proc\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.549203 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.549007 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-sys\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.549203 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.549007 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-podres\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.557619 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.557591 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl59m\" (UniqueName: \"kubernetes.io/projected/81d2a3e4-7d1d-44f3-ad90-89ace15265c2-kube-api-access-dl59m\") pod \"perf-node-gather-daemonset-5qww2\" (UID: \"81d2a3e4-7d1d-44f3-ad90-89ace15265c2\") " pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:05.655673 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.655648 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-qd7pj_9104ad2f-c7e5-48fe-ae1c-2c139d5fea4c/networking-console-plugin/0.log" Jul 07 14:46:05.727167 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:05.727066 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:06.071691 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:06.071667 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2"] Jul 07 14:46:06.073344 ip-10-0-143-146 kubenswrapper[2581]: W0707 14:46:06.073311 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod81d2a3e4_7d1d_44f3_ad90_89ace15265c2.slice/crio-09b12f7624172943306ece6c091e471f7e186cb4042f0f8492370a7d2ab25137 WatchSource:0}: Error finding container 09b12f7624172943306ece6c091e471f7e186cb4042f0f8492370a7d2ab25137: Status 404 returned error can't find the container with id 09b12f7624172943306ece6c091e471f7e186cb4042f0f8492370a7d2ab25137 Jul 07 14:46:06.296371 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:06.296342 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6rwzn_88bef153-2452-4b8c-84f4-333d8426694d/console-operator/1.log" Jul 07 14:46:06.300574 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:06.300553 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6rwzn_88bef153-2452-4b8c-84f4-333d8426694d/console-operator/2.log" Jul 07 14:46:06.856478 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:06.856448 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-vngwn_f00f710c-ea6e-407a-82f1-16fa9fb4a782/download-server/0.log" Jul 07 14:46:07.054385 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:07.054349 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" event={"ID":"81d2a3e4-7d1d-44f3-ad90-89ace15265c2","Type":"ContainerStarted","Data":"57e1dd9a4ee05e337f12286bdaf0a5c2296a974ccafbab697b2e48b05ceb6690"} Jul 07 14:46:07.054598 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:07.054440 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" event={"ID":"81d2a3e4-7d1d-44f3-ad90-89ace15265c2","Type":"ContainerStarted","Data":"09b12f7624172943306ece6c091e471f7e186cb4042f0f8492370a7d2ab25137"} Jul 07 14:46:07.054598 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:07.054503 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:07.070868 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:07.070785 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" podStartSLOduration=2.070770321 podStartE2EDuration="2.070770321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:46:07.069800825 +0000 UTC m=+772.344884252" watchObservedRunningTime="2026-07-07 14:46:07.070770321 +0000 UTC m=+772.345853745" Jul 07 14:46:07.452994 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:07.452966 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-wdldm_88545de4-3bf9-41ba-8e60-03d91dc8bd10/volume-data-source-validator/0.log" Jul 07 14:46:08.500277 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:08.500250 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2dz7j_adbb655f-b138-4258-909d-f74f718863a1/dns/0.log" Jul 07 14:46:08.545873 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:08.545844 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2dz7j_adbb655f-b138-4258-909d-f74f718863a1/kube-rbac-proxy/0.log" Jul 07 14:46:08.778660 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:08.778578 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-92t44_9ecd6676-11b6-49ad-b15f-e08fbc43c0b6/dns-node-resolver/0.log" Jul 07 14:46:09.464054 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:09.463996 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6hfkd_17ef7399-7fb7-4446-b7b5-c85cc6d7e43e/node-ca/0.log" Jul 07 14:46:10.547918 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:10.547887 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54bbbc8986-7sghp_1e2c220f-9325-4a4e-bd79-38ec5bf729c5/kube-auth-proxy/0.log" Jul 07 14:46:10.741733 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:10.741707 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-wch9q_4722d3d1-331e-41f8-91f6-d8925d707053/payload-pre-processing/0.log" Jul 07 14:46:10.809976 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:10.809890 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-vkrl7_a382906a-50f4-408c-ad1b-639740f4e590/payload-processing/0.log" Jul 07 14:46:10.833235 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:10.833199 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67bf4497f9-krpmz_2485c568-9c6b-4bfe-8a8e-9437042d1f9f/router/0.log" Jul 07 14:46:11.441049 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:11.441022 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2vgh2_7982337d-d41b-40b6-ab24-d925e4ac6196/serve-healthcheck-canary/0.log" Jul 07 14:46:11.939755 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:11.939695 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-gwq7g_dd9bcdd4-6d7b-4809-befd-7238b2ceb035/insights-operator/0.log" Jul 07 14:46:11.940393 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:11.940027 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-gwq7g_dd9bcdd4-6d7b-4809-befd-7238b2ceb035/insights-operator/1.log" Jul 07 14:46:12.105338 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:12.105314 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gznjd_8ad450ca-750b-4c5a-a5c0-bc66602536f7/kube-rbac-proxy/0.log" Jul 07 14:46:12.130210 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:12.130186 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gznjd_8ad450ca-750b-4c5a-a5c0-bc66602536f7/exporter/0.log" Jul 07 14:46:12.156051 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:12.156025 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gznjd_8ad450ca-750b-4c5a-a5c0-bc66602536f7/extractor/0.log" Jul 07 14:46:13.069520 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:13.069489 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b225k/perf-node-gather-daemonset-5qww2" Jul 07 14:46:14.454278 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:14.454250 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mx9js_130e44c8-ac80-4545-a965-67b82988c0a8/manager/0.log" Jul 07 14:46:14.489072 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:14.489038 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-64zpt_9b8f1b96-0570-4f69-a62e-940496a1b239/manager/0.log" Jul 07 14:46:14.516572 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:14.516546 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-84b56fdcc7-2vwqc_9a81072e-af15-4b07-b69d-a4d9820b67aa/maas-api/1.log" Jul 07 14:46:14.538456 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:14.538427 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-84b56fdcc7-2vwqc_9a81072e-af15-4b07-b69d-a4d9820b67aa/maas-api/0.log" Jul 07 14:46:14.595606 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:14.595579 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56bd9c557d-h8ltw_2a2803b4-1761-4af3-8ce4-a99d3c63d959/manager/0.log" Jul 07 14:46:14.631363 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:14.631337 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6hsjf_317793be-a149-4f2e-84c8-699683a38292/server/0.log" Jul 07 14:46:14.716804 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:14.716777 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cd69c54fc-gdlmw_6d06cd06-e75a-49ff-8c91-f36581f6efac/manager/0.log" Jul 07 14:46:14.748053 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:14.748027 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cd69c54fc-gdlmw_6d06cd06-e75a-49ff-8c91-f36581f6efac/copy-manifests/0.log" Jul 07 14:46:14.910962 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:14.910880 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-qlhr8_2a877257-01e2-48bc-8f90-06967c5739a4/postgres/0.log" Jul 07 14:46:16.063310 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:16.063278 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54bbbc8986-7sghp_1e2c220f-9325-4a4e-bd79-38ec5bf729c5/kube-auth-proxy/0.log" Jul 07 14:46:16.244439 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:16.244409 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-wch9q_4722d3d1-331e-41f8-91f6-d8925d707053/payload-pre-processing/0.log" Jul 07 14:46:16.295912 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:16.295882 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-vkrl7_a382906a-50f4-408c-ad1b-639740f4e590/payload-processing/0.log" Jul 07 14:46:16.310715 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:16.310694 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67bf4497f9-krpmz_2485c568-9c6b-4bfe-8a8e-9437042d1f9f/router/0.log" Jul 07 14:46:17.534748 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:17.534668 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54bbbc8986-7sghp_1e2c220f-9325-4a4e-bd79-38ec5bf729c5/kube-auth-proxy/0.log" Jul 07 14:46:17.696467 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:17.696436 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-wch9q_4722d3d1-331e-41f8-91f6-d8925d707053/payload-pre-processing/0.log" Jul 07 14:46:17.749336 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:17.749305 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-vkrl7_a382906a-50f4-408c-ad1b-639740f4e590/payload-processing/0.log" Jul 07 14:46:17.766644 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:17.766616 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67bf4497f9-krpmz_2485c568-9c6b-4bfe-8a8e-9437042d1f9f/router/0.log" Jul 07 14:46:21.216766 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:21.216736 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-72k67_c8cf2483-6838-403c-bb44-abc6adbad239/migrator/0.log" Jul 07 14:46:21.243476 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:21.243448 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-72k67_c8cf2483-6838-403c-bb44-abc6adbad239/graceful-termination/0.log" Jul 07 14:46:21.595882 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:21.595851 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-d5kgl_615e9275-9e38-4727-8c1f-98b3e94b86e1/kube-storage-version-migrator-operator/1.log" Jul 07 14:46:21.596765 ip-10-0-143-146 kubenswrapper[2581]: I0707 14:46:21.596746 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-d5kgl_615e9275-9e38-4727-8c1f-98b3e94b86e1/kube-storage-version-migrator-operator/0.log"