Jul 21 21:00:20.261829 ip-10-0-130-26 systemd[1]: Starting Kubernetes Kubelet... Jul 21 21:00:20.705898 ip-10-0-130-26 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 21:00:20.705898 ip-10-0-130-26 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 21:00:20.705898 ip-10-0-130-26 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 21:00:20.705898 ip-10-0-130-26 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 21:00:20.705898 ip-10-0-130-26 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 21:00:20.708188 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.708066 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 21:00:20.712934 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712920 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 21:00:20.712934 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712934 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712944 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712948 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712951 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712953 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712956 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712959 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712961 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712964 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712967 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712969 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712972 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712977 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712981 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712984 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712986 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712989 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712992 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712995 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.712997 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 21:00:20.712999 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713000 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713003 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713006 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713010 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713014 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713017 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713020 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713023 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713026 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713028 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713032 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713036 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713039 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713043 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713045 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713048 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713051 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713053 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713056 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 21:00:20.713467 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713058 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713061 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713064 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713067 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713070 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713072 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713075 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713077 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713080 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713082 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713085 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713087 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713090 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713092 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713095 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713098 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713101 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713103 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713106 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713109 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 21:00:20.713983 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713111 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713114 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713117 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713119 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713122 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713125 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713127 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713130 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713132 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713135 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713138 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713140 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713143 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713145 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713148 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713151 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713153 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713155 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713158 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713161 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 21:00:20.714460 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713164 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713166 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713169 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713172 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713174 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713176 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713179 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713183 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713185 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713188 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713191 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713193 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713196 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713198 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713201 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713204 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713206 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713210 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713213 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713900 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 21:00:20.714959 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713907 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713911 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713914 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713917 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713921 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713923 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713926 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713929 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713931 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713934 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713937 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713939 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713942 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713944 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713947 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713950 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713952 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713955 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713958 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713961 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 21:00:20.715472 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713964 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713966 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713969 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713971 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713974 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713977 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713980 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713982 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713985 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713988 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713991 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713993 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713996 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.713999 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714002 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714004 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714007 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714009 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714011 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714014 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 21:00:20.716006 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714016 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714019 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714021 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714024 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714027 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714031 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714034 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714037 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714055 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714059 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714063 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714068 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714071 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714074 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714077 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714080 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714082 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714085 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714089 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 21:00:20.716484 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714093 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714095 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714098 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714100 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714104 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714106 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714109 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714113 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714116 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714119 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714122 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714124 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714128 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714131 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714133 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714136 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714138 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714141 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714143 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714146 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 21:00:20.717065 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714148 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714151 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714154 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714156 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714159 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714162 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714165 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714167 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714170 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714172 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714175 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714177 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714180 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714182 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714185 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714187 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714190 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714193 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714196 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714528 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 21:00:20.718273 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714533 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714536 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714538 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714542 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714546 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714549 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714552 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714554 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714557 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714559 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714562 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714564 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714567 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714569 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714572 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714575 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714578 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714581 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714599 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714602 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 21:00:20.718785 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714605 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714608 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714610 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714613 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714616 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714619 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714621 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714624 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714626 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714630 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714633 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714635 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714638 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714640 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714643 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714645 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714648 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714650 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714653 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714656 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 21:00:20.719285 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714658 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714661 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714663 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714665 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714668 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714670 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714673 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714676 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714679 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714681 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714684 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714687 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714689 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714692 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714694 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714697 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714699 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714702 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714704 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714707 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 21:00:20.719787 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714710 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714712 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714715 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714717 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714720 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714722 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714725 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714727 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714729 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714732 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714735 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714737 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714740 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714743 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714745 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714748 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714752 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714755 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714758 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 21:00:20.720282 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714763 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714766 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714769 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714772 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714774 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714777 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714779 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714782 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714784 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714787 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714790 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714793 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714796 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714799 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714802 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714804 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714807 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714810 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.714812 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714877 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 21:00:20.720824 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714884 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714890 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714894 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714898 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714901 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714905 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714909 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714912 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714915 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714918 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714922 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714925 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714931 2568 flags.go:64] FLAG: --cgroup-root="" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714934 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714937 2568 flags.go:64] FLAG: --client-ca-file="" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714939 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714942 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714945 2568 flags.go:64] FLAG: --cluster-domain="" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714949 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714952 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714955 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714958 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714962 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 21:00:20.721291 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714965 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714968 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714971 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714974 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714977 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714980 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714984 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714986 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714990 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714993 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714996 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.714999 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715002 2568 flags.go:64] FLAG: --enable-server="true" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715005 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715009 2568 flags.go:64] FLAG: --event-burst="100" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715012 2568 flags.go:64] FLAG: --event-qps="50" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715015 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715017 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715020 2568 flags.go:64] FLAG: --eviction-hard="" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715023 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715026 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715033 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715036 2568 flags.go:64] FLAG: --eviction-soft="" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715039 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715041 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 21:00:20.721848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715044 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715047 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715050 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715053 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715055 2568 flags.go:64] FLAG: --feature-gates="" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715059 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715062 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715065 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715068 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715071 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715074 2568 flags.go:64] FLAG: --help="false" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715076 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-130-26.ec2.internal" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715079 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715082 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715086 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715089 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715092 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715095 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715098 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715101 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715103 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715106 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715109 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715112 2568 flags.go:64] FLAG: --kube-reserved="" Jul 21 21:00:20.722549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715115 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715117 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715120 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715123 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715128 2568 flags.go:64] FLAG: --lock-file="" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715131 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715134 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715137 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715142 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715145 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715148 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715150 2568 flags.go:64] FLAG: --logging-format="text" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715153 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715156 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715159 2568 flags.go:64] FLAG: --manifest-url="" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715162 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715166 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715169 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715172 2568 flags.go:64] FLAG: --max-pods="110" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715175 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715178 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715181 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715184 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715187 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715190 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 21:00:20.723232 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715202 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715210 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715213 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715215 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715219 2568 flags.go:64] FLAG: --pod-cidr="" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715221 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715226 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715229 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715232 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715235 2568 flags.go:64] FLAG: --port="10250" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715237 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715242 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0511176f654986898" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715246 2568 flags.go:64] FLAG: --qos-reserved="" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715249 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715252 2568 flags.go:64] FLAG: --register-node="true" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715254 2568 flags.go:64] FLAG: --register-with-taints="" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715258 2568 flags.go:64] FLAG: --registry-burst="10" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715261 2568 flags.go:64] FLAG: --registry-qps="5" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715263 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715266 2568 flags.go:64] FLAG: --reserved-memory="" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715269 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715272 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715275 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715278 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715280 2568 flags.go:64] FLAG: --runonce="false" Jul 21 21:00:20.724166 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715283 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715286 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715289 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715291 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715294 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715297 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715301 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715304 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715306 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715309 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715312 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715315 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715318 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715321 2568 flags.go:64] FLAG: --system-cgroups="" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715324 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715329 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715331 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715334 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715339 2568 flags.go:64] FLAG: --tls-min-version="" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715342 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715345 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715348 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715350 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715353 2568 flags.go:64] FLAG: --v="2" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715357 2568 flags.go:64] FLAG: --version="false" Jul 21 21:00:20.725083 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715360 2568 flags.go:64] FLAG: --vmodule="" Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715364 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.715367 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715471 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715474 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715477 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715480 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715484 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715487 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715489 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715492 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715494 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715497 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715500 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715503 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715506 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715508 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715512 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715516 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 21:00:20.725869 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715519 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715523 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715526 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715529 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715532 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715535 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715538 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715542 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715545 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715548 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715551 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715553 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715556 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715558 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715561 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715564 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715566 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715569 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715571 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 21:00:20.726410 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715574 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715576 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715579 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715594 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715597 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715600 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715603 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715605 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715608 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715611 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715614 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715616 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715619 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715621 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715624 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715626 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715629 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715631 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715634 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715636 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 21:00:20.727313 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715640 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715643 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715645 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715648 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715651 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715653 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715656 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715659 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715661 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715664 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715666 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715669 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715671 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715674 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715676 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715679 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715682 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715684 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715687 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715690 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 21:00:20.728032 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715692 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715694 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715697 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715700 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715703 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715707 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715710 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715713 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715715 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715718 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715721 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715724 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715728 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715731 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715734 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715736 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715739 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715742 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715744 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715747 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 21:00:20.728651 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715749 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715752 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715755 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.715757 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.716512 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.725383 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.725402 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.726261 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.726276 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726362 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726372 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726379 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726385 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726389 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 21:00:20.729216 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726395 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726399 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726404 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726409 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726413 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726417 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726421 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726425 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726429 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726433 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726438 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726444 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726458 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726463 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726468 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726472 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726476 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726480 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726485 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 21:00:20.729558 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726489 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726493 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726498 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726502 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726507 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726511 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726515 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726520 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726525 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726529 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726534 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726540 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726546 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726551 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726555 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726559 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726564 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726568 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726572 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 21:00:20.730229 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726577 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726597 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726602 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726606 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726610 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726614 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726619 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726624 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726628 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726632 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726636 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726640 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726644 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726649 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726653 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726657 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726661 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726665 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726669 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726674 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 21:00:20.731046 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726678 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726682 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726686 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726691 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726695 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726699 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726703 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726707 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726711 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726715 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726720 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726724 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726728 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726732 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726736 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726740 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726746 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726750 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726754 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726758 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 21:00:20.731537 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726762 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726766 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726771 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726775 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726779 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726783 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726787 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726791 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726795 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726799 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726803 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726814 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726819 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726823 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726827 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 21:00:20.732211 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.726831 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.726839 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727023 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727032 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727037 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727041 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727047 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727051 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727055 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727059 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727063 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727067 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727071 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727075 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 21:00:20.732726 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727083 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727089 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727095 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727099 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727103 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727110 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727115 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727119 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727123 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727128 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727133 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727138 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727142 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727146 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727150 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727154 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727158 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727162 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727166 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727170 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 21:00:20.733080 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727175 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727179 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727184 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727189 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727206 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727210 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727214 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727219 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727223 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727227 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727230 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727234 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727238 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727243 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727248 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727252 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727256 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727260 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727265 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727269 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 21:00:20.733645 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727273 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727277 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727281 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727286 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727290 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727294 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727298 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727303 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727307 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727311 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727314 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727318 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727322 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727326 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727330 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727334 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727338 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727341 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727345 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727350 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 21:00:20.734118 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727354 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727358 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727362 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727367 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727371 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727375 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727380 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727384 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727388 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727392 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727397 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727401 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727405 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727409 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727413 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727417 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727421 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727428 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727432 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 21:00:20.734673 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727437 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727441 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727445 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727450 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727454 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727458 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727464 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:20.727470 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.727478 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.727659 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.731732 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.733474 2568 server.go:1019] "Starting client certificate rotation" Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.733598 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 21:00:20.735121 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.733655 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 21:00:20.736281 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.736265 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 21:00:20.740471 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.740455 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 21:00:20.753369 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.753354 2568 log.go:25] "Validated CRI v1 runtime API" Jul 21 21:00:20.755946 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.755927 2568 log.go:25] "Validated CRI v1 image API" Jul 21 21:00:20.757668 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.757652 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 21:00:20.760226 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.760205 2568 fs.go:135] Filesystem UUIDs: map[4cd5b7cb-bc60-47a6-9392-ffd710007c2a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ff449788-fa1c-48c8-8352-fc8743ab12f9:/dev/nvme0n1p3] Jul 21 21:00:20.760274 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.760228 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 21:00:20.767077 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.766939 2568 manager.go:216] Machine: {Timestamp:2026-07-21 21:00:20.765698425 +0000 UTC m=+0.376451800 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099580 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2235ab05cf34f52bf81db50a52aea2 SystemUUID:ec2235ab-05cf-34f5-2bf8-1db50a52aea2 BootID:00d614ec-b312-456d-ae58-5e06b9a4b7bd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5c:b6:c6:95:9f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5c:b6:c6:95:9f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:e8:23:5d:95:34 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 21:00:20.767077 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.767067 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 21:00:20.767214 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.767135 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 21:00:20.767409 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.767394 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 21:00:20.768510 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.768490 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 21:00:20.769644 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.768513 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-26.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 21:00:20.771325 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.771120 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 21:00:20.771325 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.771328 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 21:00:20.771422 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.771345 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 21:00:20.772148 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.772136 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 21:00:20.774039 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.774029 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 21:00:20.778620 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.778609 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 21 21:00:20.778761 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.778751 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 21:00:20.778797 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.778782 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 21 21:00:20.778797 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.778791 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 21:00:20.778853 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.778808 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 21:00:20.778853 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.778818 2568 kubelet.go:398] "Adding apiserver pod source" Jul 21 21:00:20.778853 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.778826 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 21:00:20.779178 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.779164 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 21:00:20.779251 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.779188 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 21:00:20.781520 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.781503 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 21:00:20.783844 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.783831 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 21:00:20.783881 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.783862 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 21:00:20.785210 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785198 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 21:00:20.785262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785217 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 21:00:20.785262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785227 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 21:00:20.785262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785236 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 21:00:20.785262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785248 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 21:00:20.785262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785257 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 21:00:20.785392 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785269 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 21:00:20.785392 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785278 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 21:00:20.785392 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785288 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 21:00:20.785392 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785297 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 21:00:20.785392 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785307 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 21:00:20.785392 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.785312 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 21:00:20.787030 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.787019 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 21:00:20.787062 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.787030 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 21:00:20.790063 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.790047 2568 server.go:1268] "Started kubelet" Jul 21 21:00:20.790177 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.790150 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 21:00:20.790240 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.790205 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 21:00:20.790329 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.790199 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 21:00:20.790903 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.790547 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 21:00:20.790903 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.790698 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-26.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 21:00:20.790903 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.790700 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-26.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 21:00:20.790785 ip-10-0-130-26 systemd[1]: Started Kubernetes Kubelet. Jul 21 21:00:20.792381 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.792338 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 21:00:20.792611 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.792599 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 21 21:00:20.796225 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.796206 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 21:00:20.800754 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.800737 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 21:00:20.800937 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.799534 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-26.ec2.internal.18c469bedb54a20b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-26.ec2.internal,UID:ip-10-0-130-26.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-26.ec2.internal,},FirstTimestamp:2026-07-21 21:00:20.790018571 +0000 UTC m=+0.400771946,LastTimestamp:2026-07-21 21:00:20.790018571 +0000 UTC m=+0.400771946,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-26.ec2.internal,}" Jul 21 21:00:20.801412 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.801397 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 21:00:20.801740 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.801716 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:20.802383 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802291 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 21:00:20.802383 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802293 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 21:00:20.802383 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802328 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 21:00:20.802573 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802453 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 21 21:00:20.802573 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802465 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 21 21:00:20.802573 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802540 2568 factory.go:55] Registering systemd factory Jul 21 21:00:20.802573 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802552 2568 factory.go:223] Registration of the systemd container factory successfully Jul 21 21:00:20.802808 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802792 2568 factory.go:153] Registering CRI-O factory Jul 21 21:00:20.802864 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802811 2568 factory.go:223] Registration of the crio container factory successfully Jul 21 21:00:20.802912 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802864 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 21:00:20.802912 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802889 2568 factory.go:103] Registering Raw factory Jul 21 21:00:20.802912 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.802903 2568 manager.go:1196] Started watching for new ooms in manager Jul 21 21:00:20.803495 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.803481 2568 manager.go:349] Starting recovery of all containers Jul 21 21:00:20.810920 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.810894 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 21:00:20.810920 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.810867 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-26.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 21:00:20.811522 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.811481 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 21:00:20.814757 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.814739 2568 manager.go:354] Recovery completed Jul 21 21:00:20.817566 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.817414 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7hs7j" Jul 21 21:00:20.819183 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.819171 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 21:00:20.821449 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.821433 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory" Jul 21 21:00:20.821518 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.821464 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 21:00:20.821518 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.821475 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID" Jul 21 21:00:20.821967 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.821954 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 21:00:20.822013 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.821971 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 21:00:20.822013 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.821958 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 21:00:20.822013 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.821989 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 21 21:00:20.822611 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.822580 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory" Jul 21 21:00:20.822651 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.822619 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 21:00:20.822651 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.822629 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID" Jul 21 21:00:20.823371 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.823306 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-26.ec2.internal.18c469bedd343cbc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-26.ec2.internal,UID:ip-10-0-130-26.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-26.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-26.ec2.internal,},FirstTimestamp:2026-07-21 21:00:20.821449916 +0000 UTC m=+0.432203295,LastTimestamp:2026-07-21 21:00:20.821449916 +0000 UTC m=+0.432203295,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-26.ec2.internal,}" Jul 21 21:00:20.824149 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.824138 2568 policy_none.go:49] "None policy: Start" Jul 21 21:00:20.824181 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.824155 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 21:00:20.824181 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.824166 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 21:00:20.825233 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.825218 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7hs7j" Jul 21 21:00:20.826576 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.826561 2568 policy_none.go:47] "Start" Jul 21 21:00:20.864168 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.864154 2568 manager.go:337] "Starting Device Plugin manager" Jul 21 21:00:20.886777 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.864185 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 21:00:20.886777 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.864198 2568 server.go:87] "Starting device plugin registration server" Jul 21 21:00:20.886777 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.864549 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 21:00:20.886777 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.864559 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 21:00:20.886777 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.864686 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 21:00:20.886777 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.864768 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 21:00:20.886777 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.864777 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 21:00:20.886777 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.865241 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 21:00:20.886777 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.865301 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:20.905714 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.905697 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 21:00:20.905807 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.905733 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 21:00:20.905807 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.905753 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 21:00:20.905807 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.905784 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 21:00:20.909190 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.909173 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 21:00:20.965369 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.965318 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 21:00:20.966108 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.966091 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory" Jul 21 21:00:20.966190 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.966128 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 21:00:20.966190 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.966144 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID" Jul 21 21:00:20.966190 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.966176 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-26.ec2.internal" Jul 21 21:00:20.978268 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:20.978250 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-26.ec2.internal" Jul 21 21:00:20.978330 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:20.978271 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-26.ec2.internal\": node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:21.004851 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.004830 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:21.005849 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.005828 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal"] Jul 21 21:00:21.005918 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.005901 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 21:00:21.006821 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.006807 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory" Jul 21 21:00:21.006922 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.006840 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 21:00:21.006922 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.006855 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID" Jul 21 21:00:21.009207 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.009192 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 21:00:21.009348 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.009335 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.009392 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.009361 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 21:00:21.009986 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.009961 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory" Jul 21 21:00:21.010065 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.009992 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 21:00:21.010065 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.010002 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID" Jul 21 21:00:21.010513 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.010496 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory" Jul 21 21:00:21.010613 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.010528 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 21:00:21.010613 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.010541 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID" Jul 21 21:00:21.012753 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.012735 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.012829 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.012780 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 21:00:21.014031 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.014017 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory" Jul 21 21:00:21.014087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.014044 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 21:00:21.014087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.014054 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID" Jul 21 21:00:21.034269 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.034252 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-26.ec2.internal\" not found" node="ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.038476 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.038460 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-26.ec2.internal\" not found" node="ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.104613 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.104574 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/916d60af6bbc560eb91de9d2f508780c-config\") pod \"kube-apiserver-proxy-ip-10-0-130-26.ec2.internal\" (UID: \"916d60af6bbc560eb91de9d2f508780c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.104613 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.104612 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2efc7603de9674db809b975a7de033a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"2efc7603de9674db809b975a7de033a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.104747 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.104629 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2efc7603de9674db809b975a7de033a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"2efc7603de9674db809b975a7de033a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.105630 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.105614 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:21.205502 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.205458 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/916d60af6bbc560eb91de9d2f508780c-config\") pod \"kube-apiserver-proxy-ip-10-0-130-26.ec2.internal\" (UID: \"916d60af6bbc560eb91de9d2f508780c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.205502 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.205507 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2efc7603de9674db809b975a7de033a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"2efc7603de9674db809b975a7de033a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.205675 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.205540 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2efc7603de9674db809b975a7de033a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"2efc7603de9674db809b975a7de033a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.205675 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.205548 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/916d60af6bbc560eb91de9d2f508780c-config\") pod \"kube-apiserver-proxy-ip-10-0-130-26.ec2.internal\" (UID: \"916d60af6bbc560eb91de9d2f508780c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.205675 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.205604 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2efc7603de9674db809b975a7de033a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"2efc7603de9674db809b975a7de033a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.205675 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.205627 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2efc7603de9674db809b975a7de033a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"2efc7603de9674db809b975a7de033a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.206496 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.206481 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:21.307265 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.307234 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:21.337416 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.337396 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.341922 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.341906 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" Jul 21 21:00:21.407763 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.407725 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:21.508231 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.508207 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:21.608688 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.608623 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:21.709144 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.709118 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:21.732287 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.732260 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 21:00:21.732412 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.732392 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 21:00:21.789364 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.789338 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 21:00:21.809774 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.809751 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:21.831933 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.831899 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 20:55:21 +0000 UTC" deadline="2028-01-29 16:04:23.370016728 +0000 UTC" Jul 21 21:00:21.831933 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:21.831931 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13363h4m1.5380886s" Jul 21 21:00:21.910394 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:21.910333 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:22.010784 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:22.010760 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:22.111315 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:22.111288 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found" Jul 21 21:00:22.158317 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.156297 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 21:00:22.169156 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.169136 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 21:00:22.171107 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:22.171079 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2efc7603de9674db809b975a7de033a3.slice/crio-458389b77611351f67f56b528b5766a03882af4a4f5973527482cde911a1e08e WatchSource:0}: Error finding container 458389b77611351f67f56b528b5766a03882af4a4f5973527482cde911a1e08e: Status 404 returned error can't find the container with id 458389b77611351f67f56b528b5766a03882af4a4f5973527482cde911a1e08e Jul 21 21:00:22.171720 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:22.171697 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod916d60af6bbc560eb91de9d2f508780c.slice/crio-5353184ade4d29c2db92e0a3882fa3599832bb5f5c2e84964fa57a89d96d6151 WatchSource:0}: Error finding container 5353184ade4d29c2db92e0a3882fa3599832bb5f5c2e84964fa57a89d96d6151: Status 404 returned error can't find the container with id 5353184ade4d29c2db92e0a3882fa3599832bb5f5c2e84964fa57a89d96d6151 Jul 21 21:00:22.176593 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.176575 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 21:00:22.201685 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.201664 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" Jul 21 21:00:22.224444 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.224424 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 21:00:22.227011 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.226998 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" Jul 21 21:00:22.236460 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.236446 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 21:00:22.780055 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.780016 2568 apiserver.go:52] "Watching apiserver" Jul 21 21:00:22.788174 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.788146 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 21:00:22.788616 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.788578 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-4b7nq","openshift-dns/node-resolver-cghfw","openshift-multus/multus-additional-cni-plugins-zqk2d","openshift-network-diagnostics/network-check-target-jtn82","openshift-ovn-kubernetes/ovnkube-node-77bjb","kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal","openshift-image-registry/node-ca-2pmwz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal","openshift-multus/multus-94p6s","openshift-multus/network-metrics-daemon-ckmm7","openshift-network-operator/iptables-alerter-lsc9f","kube-system/global-pull-secret-syncer-pchv7","kube-system/konnectivity-agent-vxtql","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9"] Jul 21 21:00:22.791328 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.791305 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pmwz" Jul 21 21:00:22.797466 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.793941 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:22.797466 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.794384 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-77hx6\"" Jul 21 21:00:22.797466 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.794511 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 21:00:22.797466 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.794588 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 21:00:22.797466 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.794716 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 21:00:22.797466 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.796258 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 21:00:22.797466 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.796815 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.797466 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.796835 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 21:00:22.797466 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.797047 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zlzzs\"" Jul 21 21:00:22.798773 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.798751 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:22.798879 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:22.798833 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:22.799327 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.799306 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 21:00:22.799422 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.799352 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 21:00:22.799552 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.799532 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 21:00:22.799624 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.799586 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 21:00:22.799772 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.799595 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 21:00:22.799772 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.799707 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dcsnq\"" Jul 21 21:00:22.801454 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.801434 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 21:00:22.801589 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.801570 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.804124 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.804104 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.804350 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.804333 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 21:00:22.804448 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.804381 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k9h4q\"" Jul 21 21:00:22.804629 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.804599 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 21:00:22.804700 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.804599 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 21:00:22.804774 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.804747 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 21:00:22.804900 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.804867 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 21:00:22.804960 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.804938 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 21:00:22.806779 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.806758 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.806879 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.806780 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 21:00:22.807396 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.807287 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 21:00:22.807396 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.807298 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 21:00:22.807555 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.807338 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2rmvn\"" Jul 21 21:00:22.809367 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.809340 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zwpcz\"" Jul 21 21:00:22.809849 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.809567 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 21:00:22.811978 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.811957 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.814122 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814098 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cghfw" Jul 21 21:00:22.814276 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814253 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-cnibin\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.814341 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814295 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9bb98f87-4d9c-4654-9e10-e52bfb05591f-cni-binary-copy\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.814341 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814325 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.814456 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814349 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-ovnkube-script-lib\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814456 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814375 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f7gc\" (UniqueName: \"kubernetes.io/projected/65f51fb4-b664-45da-81ab-4a0d20506ad2-kube-api-access-8f7gc\") pod \"node-ca-2pmwz\" (UID: \"65f51fb4-b664-45da-81ab-4a0d20506ad2\") " pod="openshift-image-registry/node-ca-2pmwz" Jul 21 21:00:22.814456 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814401 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxbnf\" (UniqueName: \"kubernetes.io/projected/9bb98f87-4d9c-4654-9e10-e52bfb05591f-kube-api-access-vxbnf\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.814456 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814426 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-run-netns\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814456 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814450 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-run-ovn-kubernetes\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814656 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814478 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-ovnkube-config\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814656 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814502 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-env-overrides\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814656 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814527 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-log-socket\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814656 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814549 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a816a2e0-0ddf-42e2-ad1b-54b69582b666-agent-certs\") pod \"konnectivity-agent-vxtql\" (UID: \"a816a2e0-0ddf-42e2-ad1b-54b69582b666\") " pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:22.814656 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814573 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9bb98f87-4d9c-4654-9e10-e52bfb05591f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.814656 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814649 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814648 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-kubelet\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814716 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-ovn-node-metrics-cert\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814747 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-run-openvswitch\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814795 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-system-cni-dir\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814820 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814846 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814863 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-socket-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814877 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-slash\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814903 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-run-ovn\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814942 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-cni-netd\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.814974 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814965 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-etc-openvswitch\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.814987 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-node-log\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815022 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkjhq\" (UniqueName: \"kubernetes.io/projected/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-kube-api-access-dkjhq\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815045 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhsbp\" (UniqueName: \"kubernetes.io/projected/35e82a47-c026-420d-9e9a-fb3e524ff0b6-kube-api-access-jhsbp\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815091 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65f51fb4-b664-45da-81ab-4a0d20506ad2-host\") pod \"node-ca-2pmwz\" (UID: \"65f51fb4-b664-45da-81ab-4a0d20506ad2\") " pod="openshift-image-registry/node-ca-2pmwz" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815143 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65f51fb4-b664-45da-81ab-4a0d20506ad2-serviceca\") pod \"node-ca-2pmwz\" (UID: \"65f51fb4-b664-45da-81ab-4a0d20506ad2\") " pod="openshift-image-registry/node-ca-2pmwz" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815170 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9bb98f87-4d9c-4654-9e10-e52bfb05591f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815190 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-systemd-units\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815229 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-run-systemd\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815247 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-var-lib-openvswitch\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815262 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-registration-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815278 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-device-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815368 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a816a2e0-0ddf-42e2-ad1b-54b69582b666-konnectivity-ca\") pod \"konnectivity-agent-vxtql\" (UID: \"a816a2e0-0ddf-42e2-ad1b-54b69582b666\") " pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815425 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-os-release\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815452 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgrqg\" (UniqueName: \"kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg\") pod \"network-check-target-jtn82\" (UID: \"adc643af-b310-4c8a-a4b5-d96749bf4188\") " pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815478 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-cni-bin\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815504 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.815585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815515 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gmfpv\"" Jul 21 21:00:22.816455 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.815539 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-sys-fs\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.816511 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.816458 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 21:00:22.817694 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.817214 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:22.817694 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:22.817349 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:22.817955 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.817934 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 21:00:22.818036 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.817942 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 21:00:22.818179 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.818159 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7k5kv\"" Jul 21 21:00:22.819575 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.819498 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lsc9f" Jul 21 21:00:22.821797 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.821778 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 21:00:22.822246 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.822225 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:22.822321 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.822290 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 21:00:22.822321 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:22.822291 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:22.822321 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.822234 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5tpsj\"" Jul 21 21:00:22.823176 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.823135 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 21:00:22.846185 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.846163 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pqltc" Jul 21 21:00:22.850359 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.850340 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pqltc" Jul 21 21:00:22.903209 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.903187 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 21:00:22.910966 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.910919 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" event={"ID":"2efc7603de9674db809b975a7de033a3","Type":"ContainerStarted","Data":"458389b77611351f67f56b528b5766a03882af4a4f5973527482cde911a1e08e"} Jul 21 21:00:22.913245 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.913203 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" event={"ID":"916d60af6bbc560eb91de9d2f508780c","Type":"ContainerStarted","Data":"5353184ade4d29c2db92e0a3882fa3599832bb5f5c2e84964fa57a89d96d6151"} Jul 21 21:00:22.916637 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916586 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-cnibin\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.916749 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916643 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-multus-conf-dir\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.916749 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916683 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7f05adfe-0e27-4ea2-8e61-4e03c16fba01-hosts-file\") pod \"node-resolver-cghfw\" (UID: \"7f05adfe-0e27-4ea2-8e61-4e03c16fba01\") " pod="openshift-dns/node-resolver-cghfw" Jul 21 21:00:22.916749 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916711 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f7gc\" (UniqueName: \"kubernetes.io/projected/65f51fb4-b664-45da-81ab-4a0d20506ad2-kube-api-access-8f7gc\") pod \"node-ca-2pmwz\" (UID: \"65f51fb4-b664-45da-81ab-4a0d20506ad2\") " pod="openshift-image-registry/node-ca-2pmwz" Jul 21 21:00:22.916749 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxbnf\" (UniqueName: \"kubernetes.io/projected/9bb98f87-4d9c-4654-9e10-e52bfb05591f-kube-api-access-vxbnf\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.916972 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916759 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-run-netns\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.916972 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916791 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-run-ovn-kubernetes\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.916972 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916818 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-multus-socket-dir-parent\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.916972 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9bb98f87-4d9c-4654-9e10-e52bfb05591f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.916972 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916873 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-kubelet\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.916972 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916905 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-ovn-node-metrics-cert\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.916972 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916938 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-tmp\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.916972 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916962 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-sysconfig\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.917286 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.916988 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.917286 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917014 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-os-release\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.917286 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917040 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-run-k8s-cni-cncf-io\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.917286 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917064 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-run-netns\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.917286 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917095 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-cni-netd\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.917286 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917119 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-var-lib-kubelet\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.917568 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917547 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-etc-openvswitch\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.917683 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917572 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkjhq\" (UniqueName: \"kubernetes.io/projected/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-kube-api-access-dkjhq\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.917683 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917593 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-sysctl-d\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.917683 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917615 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-sys\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.917683 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917638 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-lib-modules\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.917683 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917657 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-tuned\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.917949 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917695 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-multus-cni-dir\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.917949 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917720 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7g24\" (UniqueName: \"kubernetes.io/projected/cd4e8142-0742-404d-a5d1-f21f20672947-kube-api-access-c7g24\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.917949 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917746 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65f51fb4-b664-45da-81ab-4a0d20506ad2-serviceca\") pod \"node-ca-2pmwz\" (UID: \"65f51fb4-b664-45da-81ab-4a0d20506ad2\") " pod="openshift-image-registry/node-ca-2pmwz" Jul 21 21:00:22.917949 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917770 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-systemd-units\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.917949 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-run-systemd\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.917949 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.917949 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917869 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-sys-fs\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.917949 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917894 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-systemd\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.917949 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917920 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-var-lib-cni-multus\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.917949 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917945 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph9qs\" (UniqueName: \"kubernetes.io/projected/7f05adfe-0e27-4ea2-8e61-4e03c16fba01-kube-api-access-ph9qs\") pod \"node-resolver-cghfw\" (UID: \"7f05adfe-0e27-4ea2-8e61-4e03c16fba01\") " pod="openshift-dns/node-resolver-cghfw" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917973 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a816a2e0-0ddf-42e2-ad1b-54b69582b666-konnectivity-ca\") pod \"konnectivity-agent-vxtql\" (UID: \"a816a2e0-0ddf-42e2-ad1b-54b69582b666\") " pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.917999 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-os-release\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918023 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-cni-bin\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918048 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f16baa81-ef27-43dc-9892-58ab92744f43-host-slash\") pod \"iptables-alerter-lsc9f\" (UID: \"f16baa81-ef27-43dc-9892-58ab92744f43\") " pod="openshift-network-operator/iptables-alerter-lsc9f" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918072 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czcjh\" (UniqueName: \"kubernetes.io/projected/f16baa81-ef27-43dc-9892-58ab92744f43-kube-api-access-czcjh\") pod \"iptables-alerter-lsc9f\" (UID: \"f16baa81-ef27-43dc-9892-58ab92744f43\") " pod="openshift-network-operator/iptables-alerter-lsc9f" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918098 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9bb98f87-4d9c-4654-9e10-e52bfb05591f-cni-binary-copy\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918125 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-ovnkube-script-lib\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918175 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-run\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918198 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rmtj\" (UniqueName: \"kubernetes.io/projected/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-kube-api-access-9rmtj\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918222 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f05adfe-0e27-4ea2-8e61-4e03c16fba01-tmp-dir\") pod \"node-resolver-cghfw\" (UID: \"7f05adfe-0e27-4ea2-8e61-4e03c16fba01\") " pod="openshift-dns/node-resolver-cghfw" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918247 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-ovnkube-config\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918272 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-env-overrides\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918297 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-log-socket\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.918340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918322 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a816a2e0-0ddf-42e2-ad1b-54b69582b666-agent-certs\") pod \"konnectivity-agent-vxtql\" (UID: \"a816a2e0-0ddf-42e2-ad1b-54b69582b666\") " pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918346 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-sysctl-conf\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918376 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-system-cni-dir\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918400 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-hostroot\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918425 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918451 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-run-openvswitch\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918477 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-system-cni-dir\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918504 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918530 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-socket-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918554 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-modprobe-d\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918578 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd4e8142-0742-404d-a5d1-f21f20672947-cni-binary-copy\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918603 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd4e8142-0742-404d-a5d1-f21f20672947-multus-daemon-config\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918630 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-slash\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918681 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-run-ovn\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918707 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-run-multus-certs\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918732 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv2l4\" (UniqueName: \"kubernetes.io/projected/819c1153-e1db-4114-8e57-eb79d24d8db9-kube-api-access-tv2l4\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918759 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f16baa81-ef27-43dc-9892-58ab92744f43-iptables-alerter-script\") pod \"iptables-alerter-lsc9f\" (UID: \"f16baa81-ef27-43dc-9892-58ab92744f43\") " pod="openshift-network-operator/iptables-alerter-lsc9f" Jul 21 21:00:22.919014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918782 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/681e613d-9c45-4255-be57-f765bec9fb14-dbus\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918807 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-node-log\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918847 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhsbp\" (UniqueName: \"kubernetes.io/projected/35e82a47-c026-420d-9e9a-fb3e524ff0b6-kube-api-access-jhsbp\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918872 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-cnibin\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-var-lib-cni-bin\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918923 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-var-lib-kubelet\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918947 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-etc-kubernetes\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918972 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.918999 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65f51fb4-b664-45da-81ab-4a0d20506ad2-host\") pod \"node-ca-2pmwz\" (UID: \"65f51fb4-b664-45da-81ab-4a0d20506ad2\") " pod="openshift-image-registry/node-ca-2pmwz" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.919025 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9bb98f87-4d9c-4654-9e10-e52bfb05591f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.919052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-var-lib-openvswitch\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.919079 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-registration-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.919105 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-device-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.919129 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-kubernetes\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.919153 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-host\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.919182 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrqg\" (UniqueName: \"kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg\") pod \"network-check-target-jtn82\" (UID: \"adc643af-b310-4c8a-a4b5-d96749bf4188\") " pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.919207 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/681e613d-9c45-4255-be57-f765bec9fb14-kubelet-config\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:22.919738 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.919307 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-cnibin\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.920443 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.919808 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-run-netns\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.920443 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.919856 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-run-ovn-kubernetes\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.920443 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.920423 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9bb98f87-4d9c-4654-9e10-e52bfb05591f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.920566 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.920499 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-kubelet\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.920839 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.920817 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 21:00:22.921803 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.921772 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-ovnkube-config\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.921916 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.921903 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.922005 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.921990 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-cni-netd\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.922057 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.922041 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-etc-openvswitch\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.929188 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.929156 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-ovn-node-metrics-cert\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.929288 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.929246 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-env-overrides\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.929354 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.929316 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-log-socket\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.929872 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.929846 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-node-log\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.930159 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.930134 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65f51fb4-b664-45da-81ab-4a0d20506ad2-host\") pod \"node-ca-2pmwz\" (UID: \"65f51fb4-b664-45da-81ab-4a0d20506ad2\") " pod="openshift-image-registry/node-ca-2pmwz" Jul 21 21:00:22.930286 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.930254 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-device-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.930362 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.930297 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-run-openvswitch\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.930362 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.930325 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-var-lib-openvswitch\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.930472 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.930360 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-system-cni-dir\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.930472 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.930384 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-registration-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.930587 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.930506 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-socket-dir\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.930667 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.930606 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-run-ovn\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.930735 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.930653 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9bb98f87-4d9c-4654-9e10-e52bfb05591f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.930735 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.930688 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-slash\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.930940 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.930916 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-systemd-units\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.931104 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.931078 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-run-systemd\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.931284 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.931260 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65f51fb4-b664-45da-81ab-4a0d20506ad2-serviceca\") pod \"node-ca-2pmwz\" (UID: \"65f51fb4-b664-45da-81ab-4a0d20506ad2\") " pod="openshift-image-registry/node-ca-2pmwz" Jul 21 21:00:22.931882 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.931845 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.931981 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.931933 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.932234 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.932210 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.932315 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.932300 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35e82a47-c026-420d-9e9a-fb3e524ff0b6-sys-fs\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:22.932426 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.932389 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9bb98f87-4d9c-4654-9e10-e52bfb05591f-os-release\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.932508 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.932478 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9bb98f87-4d9c-4654-9e10-e52bfb05591f-cni-binary-copy\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.932508 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.932484 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-host-cni-bin\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.933075 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.932835 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-ovnkube-script-lib\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.933416 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.933067 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a816a2e0-0ddf-42e2-ad1b-54b69582b666-konnectivity-ca\") pod \"konnectivity-agent-vxtql\" (UID: \"a816a2e0-0ddf-42e2-ad1b-54b69582b666\") " pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:22.933653 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.933634 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a816a2e0-0ddf-42e2-ad1b-54b69582b666-agent-certs\") pod \"konnectivity-agent-vxtql\" (UID: \"a816a2e0-0ddf-42e2-ad1b-54b69582b666\") " pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:22.933749 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.933726 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxbnf\" (UniqueName: \"kubernetes.io/projected/9bb98f87-4d9c-4654-9e10-e52bfb05591f-kube-api-access-vxbnf\") pod \"multus-additional-cni-plugins-zqk2d\" (UID: \"9bb98f87-4d9c-4654-9e10-e52bfb05591f\") " pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:22.934663 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.934641 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f7gc\" (UniqueName: \"kubernetes.io/projected/65f51fb4-b664-45da-81ab-4a0d20506ad2-kube-api-access-8f7gc\") pod \"node-ca-2pmwz\" (UID: \"65f51fb4-b664-45da-81ab-4a0d20506ad2\") " pod="openshift-image-registry/node-ca-2pmwz" Jul 21 21:00:22.935543 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.935521 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkjhq\" (UniqueName: \"kubernetes.io/projected/4ef7d53f-cca5-4dc9-9c36-d064660f3f99-kube-api-access-dkjhq\") pod \"ovnkube-node-77bjb\" (UID: \"4ef7d53f-cca5-4dc9-9c36-d064660f3f99\") " pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:22.936940 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:22.936925 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 21:00:22.937123 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:22.937088 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 21:00:22.937290 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:22.937280 2568 projected.go:196] Error preparing data for projected volume kube-api-access-qgrqg for pod openshift-network-diagnostics/network-check-target-jtn82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:22.937500 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:22.937489 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg podName:adc643af-b310-4c8a-a4b5-d96749bf4188 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:23.437409583 +0000 UTC m=+3.048163003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qgrqg" (UniqueName: "kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg") pod "network-check-target-jtn82" (UID: "adc643af-b310-4c8a-a4b5-d96749bf4188") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:22.939164 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:22.939112 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhsbp\" (UniqueName: \"kubernetes.io/projected/35e82a47-c026-420d-9e9a-fb3e524ff0b6-kube-api-access-jhsbp\") pod \"aws-ebs-csi-driver-node-c6sg9\" (UID: \"35e82a47-c026-420d-9e9a-fb3e524ff0b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:23.020327 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020286 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-cnibin\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020327 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-var-lib-cni-bin\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020553 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020353 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-var-lib-kubelet\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020553 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020430 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-var-lib-cni-bin\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020553 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020446 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-cnibin\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020553 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020493 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-var-lib-kubelet\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020553 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020522 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-etc-kubernetes\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020553 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020549 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020581 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-kubernetes\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020608 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-etc-kubernetes\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020628 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-host\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020668 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/681e613d-9c45-4255-be57-f765bec9fb14-kubelet-config\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020692 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-multus-conf-dir\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020717 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7f05adfe-0e27-4ea2-8e61-4e03c16fba01-hosts-file\") pod \"node-resolver-cghfw\" (UID: \"7f05adfe-0e27-4ea2-8e61-4e03c16fba01\") " pod="openshift-dns/node-resolver-cghfw" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020748 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-multus-socket-dir-parent\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020764 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-host\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-tmp\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020799 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-sysconfig\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020818 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-kubernetes\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.020719 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020828 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-os-release\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020854 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-run-k8s-cni-cncf-io\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-run-netns\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.020895 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.020901 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret podName:681e613d-9c45-4255-be57-f765bec9fb14 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:23.520881326 +0000 UTC m=+3.131634717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret") pod "global-pull-secret-syncer-pchv7" (UID: "681e613d-9c45-4255-be57-f765bec9fb14") : object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020918 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-run-netns\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020928 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-var-lib-kubelet\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020949 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-sysctl-d\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020965 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-sys\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020976 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-multus-socket-dir-parent\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021011 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/681e613d-9c45-4255-be57-f765bec9fb14-kubelet-config\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021038 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-multus-conf-dir\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021068 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7f05adfe-0e27-4ea2-8e61-4e03c16fba01-hosts-file\") pod \"node-resolver-cghfw\" (UID: \"7f05adfe-0e27-4ea2-8e61-4e03c16fba01\") " pod="openshift-dns/node-resolver-cghfw" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021102 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-lib-modules\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021246 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-var-lib-kubelet\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021331 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-sysctl-d\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021360 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-sys\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021434 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-os-release\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021443 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-run-k8s-cni-cncf-io\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.020981 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-lib-modules\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021482 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-sysconfig\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021507 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-tuned\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.021811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021537 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-multus-cni-dir\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021565 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7g24\" (UniqueName: \"kubernetes.io/projected/cd4e8142-0742-404d-a5d1-f21f20672947-kube-api-access-c7g24\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021741 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-multus-cni-dir\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021812 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-systemd\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-var-lib-cni-multus\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021872 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph9qs\" (UniqueName: \"kubernetes.io/projected/7f05adfe-0e27-4ea2-8e61-4e03c16fba01-kube-api-access-ph9qs\") pod \"node-resolver-cghfw\" (UID: \"7f05adfe-0e27-4ea2-8e61-4e03c16fba01\") " pod="openshift-dns/node-resolver-cghfw" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f16baa81-ef27-43dc-9892-58ab92744f43-host-slash\") pod \"iptables-alerter-lsc9f\" (UID: \"f16baa81-ef27-43dc-9892-58ab92744f43\") " pod="openshift-network-operator/iptables-alerter-lsc9f" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021914 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-systemd\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.021948 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czcjh\" (UniqueName: \"kubernetes.io/projected/f16baa81-ef27-43dc-9892-58ab92744f43-kube-api-access-czcjh\") pod \"iptables-alerter-lsc9f\" (UID: \"f16baa81-ef27-43dc-9892-58ab92744f43\") " pod="openshift-network-operator/iptables-alerter-lsc9f" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022000 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-run\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022018 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f16baa81-ef27-43dc-9892-58ab92744f43-host-slash\") pod \"iptables-alerter-lsc9f\" (UID: \"f16baa81-ef27-43dc-9892-58ab92744f43\") " pod="openshift-network-operator/iptables-alerter-lsc9f" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022106 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-run\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022027 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rmtj\" (UniqueName: \"kubernetes.io/projected/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-kube-api-access-9rmtj\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022150 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f05adfe-0e27-4ea2-8e61-4e03c16fba01-tmp-dir\") pod \"node-resolver-cghfw\" (UID: \"7f05adfe-0e27-4ea2-8e61-4e03c16fba01\") " pod="openshift-dns/node-resolver-cghfw" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022182 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-sysctl-conf\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022208 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-system-cni-dir\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022249 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-hostroot\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022274 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:23.022792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022306 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-modprobe-d\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd4e8142-0742-404d-a5d1-f21f20672947-cni-binary-copy\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022387 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd4e8142-0742-404d-a5d1-f21f20672947-multus-daemon-config\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022426 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-var-lib-cni-multus\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022441 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-run-multus-certs\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022488 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-host-run-multus-certs\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv2l4\" (UniqueName: \"kubernetes.io/projected/819c1153-e1db-4114-8e57-eb79d24d8db9-kube-api-access-tv2l4\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022542 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f16baa81-ef27-43dc-9892-58ab92744f43-iptables-alerter-script\") pod \"iptables-alerter-lsc9f\" (UID: \"f16baa81-ef27-43dc-9892-58ab92744f43\") " pod="openshift-network-operator/iptables-alerter-lsc9f" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022572 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/681e613d-9c45-4255-be57-f765bec9fb14-dbus\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022624 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-sysctl-conf\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022703 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-system-cni-dir\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022750 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/681e613d-9c45-4255-be57-f765bec9fb14-dbus\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022509 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f05adfe-0e27-4ea2-8e61-4e03c16fba01-tmp-dir\") pod \"node-resolver-cghfw\" (UID: \"7f05adfe-0e27-4ea2-8e61-4e03c16fba01\") " pod="openshift-dns/node-resolver-cghfw" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.022815 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-modprobe-d\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.023353 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd4e8142-0742-404d-a5d1-f21f20672947-cni-binary-copy\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.023420 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd4e8142-0742-404d-a5d1-f21f20672947-hostroot\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.023433 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd4e8142-0742-404d-a5d1-f21f20672947-multus-daemon-config\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.023548 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:23.023615 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.023625 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs podName:819c1153-e1db-4114-8e57-eb79d24d8db9 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:23.523608244 +0000 UTC m=+3.134361608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs") pod "network-metrics-daemon-ckmm7" (UID: "819c1153-e1db-4114-8e57-eb79d24d8db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:23.024400 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.024061 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f16baa81-ef27-43dc-9892-58ab92744f43-iptables-alerter-script\") pod \"iptables-alerter-lsc9f\" (UID: \"f16baa81-ef27-43dc-9892-58ab92744f43\") " pod="openshift-network-operator/iptables-alerter-lsc9f" Jul 21 21:00:23.024400 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.024252 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-tmp\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.025876 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.024538 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-etc-tuned\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.042728 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.042426 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7g24\" (UniqueName: \"kubernetes.io/projected/cd4e8142-0742-404d-a5d1-f21f20672947-kube-api-access-c7g24\") pod \"multus-94p6s\" (UID: \"cd4e8142-0742-404d-a5d1-f21f20672947\") " pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.042728 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.042443 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv2l4\" (UniqueName: \"kubernetes.io/projected/819c1153-e1db-4114-8e57-eb79d24d8db9-kube-api-access-tv2l4\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:23.044776 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.044753 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rmtj\" (UniqueName: \"kubernetes.io/projected/0ad7a23a-99f7-4e57-8ebd-5f1513ffae20-kube-api-access-9rmtj\") pod \"tuned-4b7nq\" (UID: \"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20\") " pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.045350 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.045325 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czcjh\" (UniqueName: \"kubernetes.io/projected/f16baa81-ef27-43dc-9892-58ab92744f43-kube-api-access-czcjh\") pod \"iptables-alerter-lsc9f\" (UID: \"f16baa81-ef27-43dc-9892-58ab92744f43\") " pod="openshift-network-operator/iptables-alerter-lsc9f" Jul 21 21:00:23.046307 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.046280 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph9qs\" (UniqueName: \"kubernetes.io/projected/7f05adfe-0e27-4ea2-8e61-4e03c16fba01-kube-api-access-ph9qs\") pod \"node-resolver-cghfw\" (UID: \"7f05adfe-0e27-4ea2-8e61-4e03c16fba01\") " pod="openshift-dns/node-resolver-cghfw" Jul 21 21:00:23.104713 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.104679 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pmwz" Jul 21 21:00:23.113708 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.113685 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:23.115317 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:23.115291 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f51fb4_b664_45da_81ab_4a0d20506ad2.slice/crio-c2b7c3fa3db1f9948ef75576ce5aa0f1a1396e640d66dde2ced61b6f999e5aa5 WatchSource:0}: Error finding container c2b7c3fa3db1f9948ef75576ce5aa0f1a1396e640d66dde2ced61b6f999e5aa5: Status 404 returned error can't find the container with id c2b7c3fa3db1f9948ef75576ce5aa0f1a1396e640d66dde2ced61b6f999e5aa5 Jul 21 21:00:23.124253 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.124061 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zqk2d" Jul 21 21:00:23.127801 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:23.127672 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda816a2e0_0ddf_42e2_ad1b_54b69582b666.slice/crio-d1d63f439dd7ad5e2426c73ed62073600fcf6bb872f184e84c4b12e10e864cc8 WatchSource:0}: Error finding container d1d63f439dd7ad5e2426c73ed62073600fcf6bb872f184e84c4b12e10e864cc8: Status 404 returned error can't find the container with id d1d63f439dd7ad5e2426c73ed62073600fcf6bb872f184e84c4b12e10e864cc8 Jul 21 21:00:23.131062 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.131040 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:23.134372 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:23.134347 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb98f87_4d9c_4654_9e10_e52bfb05591f.slice/crio-f4512537cede7654f6daf3ca5846bb0c4f91c1d31d173c9bc6735e55806a2f98 WatchSource:0}: Error finding container f4512537cede7654f6daf3ca5846bb0c4f91c1d31d173c9bc6735e55806a2f98: Status 404 returned error can't find the container with id f4512537cede7654f6daf3ca5846bb0c4f91c1d31d173c9bc6735e55806a2f98 Jul 21 21:00:23.140152 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.140124 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" Jul 21 21:00:23.141260 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:23.141237 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ef7d53f_cca5_4dc9_9c36_d064660f3f99.slice/crio-62e67015b54c458a109b5bd709d81f580dbe353cd2573db18ffe3e797e46c0e2 WatchSource:0}: Error finding container 62e67015b54c458a109b5bd709d81f580dbe353cd2573db18ffe3e797e46c0e2: Status 404 returned error can't find the container with id 62e67015b54c458a109b5bd709d81f580dbe353cd2573db18ffe3e797e46c0e2 Jul 21 21:00:23.150322 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.150286 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-94p6s" Jul 21 21:00:23.151067 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:23.151044 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e82a47_c026_420d_9e9a_fb3e524ff0b6.slice/crio-8b597f4013e79e960a1d554ac960ccb548566d24e71ae129d4710df2bb560331 WatchSource:0}: Error finding container 8b597f4013e79e960a1d554ac960ccb548566d24e71ae129d4710df2bb560331: Status 404 returned error can't find the container with id 8b597f4013e79e960a1d554ac960ccb548566d24e71ae129d4710df2bb560331 Jul 21 21:00:23.158095 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.158071 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" Jul 21 21:00:23.161632 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:23.161606 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd4e8142_0742_404d_a5d1_f21f20672947.slice/crio-58f33e7b48dce5716efe17c2e4623bfdfeb9f3382ea4551af4e7161720fdd8fb WatchSource:0}: Error finding container 58f33e7b48dce5716efe17c2e4623bfdfeb9f3382ea4551af4e7161720fdd8fb: Status 404 returned error can't find the container with id 58f33e7b48dce5716efe17c2e4623bfdfeb9f3382ea4551af4e7161720fdd8fb Jul 21 21:00:23.164502 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.164486 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cghfw" Jul 21 21:00:23.169126 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:23.169091 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad7a23a_99f7_4e57_8ebd_5f1513ffae20.slice/crio-bd78bb7bac44a2135d089473491fa6d0d09c8dda6c97ae77fc69a277426ce752 WatchSource:0}: Error finding container bd78bb7bac44a2135d089473491fa6d0d09c8dda6c97ae77fc69a277426ce752: Status 404 returned error can't find the container with id bd78bb7bac44a2135d089473491fa6d0d09c8dda6c97ae77fc69a277426ce752 Jul 21 21:00:23.172195 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.172172 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lsc9f" Jul 21 21:00:23.183823 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:23.183784 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16baa81_ef27_43dc_9892_58ab92744f43.slice/crio-c37b49a8f528cc6693071823181410bc111a991d38aa17b74ee6e0c51ccde222 WatchSource:0}: Error finding container c37b49a8f528cc6693071823181410bc111a991d38aa17b74ee6e0c51ccde222: Status 404 returned error can't find the container with id c37b49a8f528cc6693071823181410bc111a991d38aa17b74ee6e0c51ccde222 Jul 21 21:00:23.327682 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.327651 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 21:00:23.526112 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.526069 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:23.526285 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.526128 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:23.526285 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.526166 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrqg\" (UniqueName: \"kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg\") pod \"network-check-target-jtn82\" (UID: \"adc643af-b310-4c8a-a4b5-d96749bf4188\") " pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:23.526285 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.526269 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:23.526440 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.526299 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 21:00:23.526440 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.526319 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 21:00:23.526440 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.526334 2568 projected.go:196] Error preparing data for projected volume kube-api-access-qgrqg for pod openshift-network-diagnostics/network-check-target-jtn82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:23.526440 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.526346 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs podName:819c1153-e1db-4114-8e57-eb79d24d8db9 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:24.526331061 +0000 UTC m=+4.137084430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs") pod "network-metrics-daemon-ckmm7" (UID: "819c1153-e1db-4114-8e57-eb79d24d8db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:23.526440 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.526381 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg podName:adc643af-b310-4c8a-a4b5-d96749bf4188 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:24.526369513 +0000 UTC m=+4.137122877 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgrqg" (UniqueName: "kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg") pod "network-check-target-jtn82" (UID: "adc643af-b310-4c8a-a4b5-d96749bf4188") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:23.526686 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.526450 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:23.526686 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:23.526486 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret podName:681e613d-9c45-4255-be57-f765bec9fb14 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:24.526477377 +0000 UTC m=+4.137230740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret") pod "global-pull-secret-syncer-pchv7" (UID: "681e613d-9c45-4255-be57-f765bec9fb14") : object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:23.854313 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.854270 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 20:55:22 +0000 UTC" deadline="2028-01-22 20:18:45.122692041 +0000 UTC" Jul 21 21:00:23.854313 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.854307 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13199h18m21.268390144s" Jul 21 21:00:23.916922 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.916879 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqk2d" event={"ID":"9bb98f87-4d9c-4654-9e10-e52bfb05591f","Type":"ContainerStarted","Data":"f4512537cede7654f6daf3ca5846bb0c4f91c1d31d173c9bc6735e55806a2f98"} Jul 21 21:00:23.918282 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.918253 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pmwz" event={"ID":"65f51fb4-b664-45da-81ab-4a0d20506ad2","Type":"ContainerStarted","Data":"c2b7c3fa3db1f9948ef75576ce5aa0f1a1396e640d66dde2ced61b6f999e5aa5"} Jul 21 21:00:23.919603 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.919579 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-94p6s" event={"ID":"cd4e8142-0742-404d-a5d1-f21f20672947","Type":"ContainerStarted","Data":"58f33e7b48dce5716efe17c2e4623bfdfeb9f3382ea4551af4e7161720fdd8fb"} Jul 21 21:00:23.920947 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.920909 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vxtql" event={"ID":"a816a2e0-0ddf-42e2-ad1b-54b69582b666","Type":"ContainerStarted","Data":"d1d63f439dd7ad5e2426c73ed62073600fcf6bb872f184e84c4b12e10e864cc8"} Jul 21 21:00:23.922239 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.922213 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lsc9f" event={"ID":"f16baa81-ef27-43dc-9892-58ab92744f43","Type":"ContainerStarted","Data":"c37b49a8f528cc6693071823181410bc111a991d38aa17b74ee6e0c51ccde222"} Jul 21 21:00:23.923443 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.923413 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cghfw" event={"ID":"7f05adfe-0e27-4ea2-8e61-4e03c16fba01","Type":"ContainerStarted","Data":"e19572ebd4f6fe7aaa863116bf3117f006efb4dd2c559bec1a3b82c7fe9c4a5d"} Jul 21 21:00:23.924875 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.924827 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" event={"ID":"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20","Type":"ContainerStarted","Data":"bd78bb7bac44a2135d089473491fa6d0d09c8dda6c97ae77fc69a277426ce752"} Jul 21 21:00:23.926266 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.926245 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" event={"ID":"35e82a47-c026-420d-9e9a-fb3e524ff0b6","Type":"ContainerStarted","Data":"8b597f4013e79e960a1d554ac960ccb548566d24e71ae129d4710df2bb560331"} Jul 21 21:00:23.927679 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:23.927651 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" event={"ID":"4ef7d53f-cca5-4dc9-9c36-d064660f3f99","Type":"ContainerStarted","Data":"62e67015b54c458a109b5bd709d81f580dbe353cd2573db18ffe3e797e46c0e2"} Jul 21 21:00:24.535706 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:24.535676 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:24.535807 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:24.535723 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:24.535807 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:24.535749 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrqg\" (UniqueName: \"kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg\") pod \"network-check-target-jtn82\" (UID: \"adc643af-b310-4c8a-a4b5-d96749bf4188\") " pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:24.535942 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:24.535833 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:24.535942 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:24.535837 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:24.535942 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:24.535876 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret podName:681e613d-9c45-4255-be57-f765bec9fb14 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:26.53586365 +0000 UTC m=+6.146617013 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret") pod "global-pull-secret-syncer-pchv7" (UID: "681e613d-9c45-4255-be57-f765bec9fb14") : object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:24.535942 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:24.535895 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs podName:819c1153-e1db-4114-8e57-eb79d24d8db9 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:26.535885474 +0000 UTC m=+6.146638837 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs") pod "network-metrics-daemon-ckmm7" (UID: "819c1153-e1db-4114-8e57-eb79d24d8db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:24.536169 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:24.535982 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 21:00:24.536169 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:24.535994 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 21:00:24.536169 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:24.536003 2568 projected.go:196] Error preparing data for projected volume kube-api-access-qgrqg for pod openshift-network-diagnostics/network-check-target-jtn82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:24.536169 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:24.536045 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg podName:adc643af-b310-4c8a-a4b5-d96749bf4188 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:26.536027965 +0000 UTC m=+6.146781342 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgrqg" (UniqueName: "kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg") pod "network-check-target-jtn82" (UID: "adc643af-b310-4c8a-a4b5-d96749bf4188") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:24.855585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:24.855304 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 20:55:22 +0000 UTC" deadline="2028-05-06 00:00:18.818206972 +0000 UTC" Jul 21 21:00:24.855585 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:24.855585 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15698h59m53.962628203s" Jul 21 21:00:24.909521 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:24.908674 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:24.909521 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:24.908815 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:24.909521 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:24.909213 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:24.909521 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:24.909302 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:24.909521 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:24.909359 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:24.909521 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:24.909430 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:24.947400 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:24.946779 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" event={"ID":"916d60af6bbc560eb91de9d2f508780c","Type":"ContainerStarted","Data":"21e3ef0b870eab61f5c5373719deca36bb254bc6913af9e09d838a90082be3b1"} Jul 21 21:00:24.964066 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:24.964011 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" podStartSLOduration=2.963991762 podStartE2EDuration="2.963991762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:24.963649464 +0000 UTC m=+4.574402848" watchObservedRunningTime="2026-07-21 21:00:24.963991762 +0000 UTC m=+4.574745150" Jul 21 21:00:25.956761 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:25.956467 2568 generic.go:356] "Generic (PLEG): container finished" podID="2efc7603de9674db809b975a7de033a3" containerID="fb30e08c468fbe02bc513bd9527321e2ec82612fd50fcda460a80a227e7c51c1" exitCode=0 Jul 21 21:00:25.956761 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:25.956671 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" event={"ID":"2efc7603de9674db809b975a7de033a3","Type":"ContainerDied","Data":"fb30e08c468fbe02bc513bd9527321e2ec82612fd50fcda460a80a227e7c51c1"} Jul 21 21:00:26.556830 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:26.556796 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:26.557011 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:26.556841 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:26.557011 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:26.556862 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrqg\" (UniqueName: \"kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg\") pod \"network-check-target-jtn82\" (UID: \"adc643af-b310-4c8a-a4b5-d96749bf4188\") " pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:26.557011 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:26.556989 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 21:00:26.557011 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:26.557002 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 21:00:26.557011 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:26.557011 2568 projected.go:196] Error preparing data for projected volume kube-api-access-qgrqg for pod openshift-network-diagnostics/network-check-target-jtn82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:26.557283 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:26.557053 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg podName:adc643af-b310-4c8a-a4b5-d96749bf4188 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:30.557039967 +0000 UTC m=+10.167793329 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgrqg" (UniqueName: "kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg") pod "network-check-target-jtn82" (UID: "adc643af-b310-4c8a-a4b5-d96749bf4188") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:26.557352 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:26.557331 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:26.557402 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:26.557363 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs podName:819c1153-e1db-4114-8e57-eb79d24d8db9 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:30.557354017 +0000 UTC m=+10.168107379 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs") pod "network-metrics-daemon-ckmm7" (UID: "819c1153-e1db-4114-8e57-eb79d24d8db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:26.557402 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:26.557395 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:26.557502 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:26.557419 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret podName:681e613d-9c45-4255-be57-f765bec9fb14 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:30.557409797 +0000 UTC m=+10.168163158 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret") pod "global-pull-secret-syncer-pchv7" (UID: "681e613d-9c45-4255-be57-f765bec9fb14") : object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:26.916911 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:26.916828 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:26.917079 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:26.916946 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:26.917307 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:26.917286 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:26.917568 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:26.917401 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:26.917568 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:26.917451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:26.917568 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:26.917521 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:26.983386 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:26.983349 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" event={"ID":"2efc7603de9674db809b975a7de033a3","Type":"ContainerStarted","Data":"c878c406597be5768693385034ce3c8bedaa44847f54323e59e11c33f3f81aee"} Jul 21 21:00:28.912137 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:28.911315 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:28.912137 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:28.911436 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:28.912137 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:28.911851 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:28.912137 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:28.911952 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:28.912137 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:28.912005 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:28.912137 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:28.912075 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:30.593060 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:30.593021 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:30.593492 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:30.593081 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:30.593492 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:30.593117 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrqg\" (UniqueName: \"kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg\") pod \"network-check-target-jtn82\" (UID: \"adc643af-b310-4c8a-a4b5-d96749bf4188\") " pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:30.593492 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:30.593212 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:30.593492 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:30.593265 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 21:00:30.593492 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:30.593284 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 21:00:30.593492 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:30.593287 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret podName:681e613d-9c45-4255-be57-f765bec9fb14 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:38.593266288 +0000 UTC m=+18.204019663 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret") pod "global-pull-secret-syncer-pchv7" (UID: "681e613d-9c45-4255-be57-f765bec9fb14") : object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:30.593492 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:30.593298 2568 projected.go:196] Error preparing data for projected volume kube-api-access-qgrqg for pod openshift-network-diagnostics/network-check-target-jtn82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:30.593492 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:30.593212 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:30.593492 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:30.593348 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg podName:adc643af-b310-4c8a-a4b5-d96749bf4188 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:38.593332883 +0000 UTC m=+18.204086248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgrqg" (UniqueName: "kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg") pod "network-check-target-jtn82" (UID: "adc643af-b310-4c8a-a4b5-d96749bf4188") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:30.593492 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:30.593366 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs podName:819c1153-e1db-4114-8e57-eb79d24d8db9 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:38.593356665 +0000 UTC m=+18.204110032 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs") pod "network-metrics-daemon-ckmm7" (UID: "819c1153-e1db-4114-8e57-eb79d24d8db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:30.906274 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:30.906202 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:30.906430 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:30.906293 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:30.906632 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:30.906615 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:30.906706 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:30.906695 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:30.906741 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:30.906726 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:30.906785 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:30.906768 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:32.910491 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:32.909799 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:32.910491 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:32.909937 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:32.910491 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:32.910021 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:32.910491 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:32.910110 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:32.910491 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:32.910165 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:32.910491 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:32.910210 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:34.909267 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:34.909233 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:34.909682 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:34.909272 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:34.909682 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:34.909406 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:34.909682 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:34.909403 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:34.909682 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:34.909561 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:34.909885 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:34.909702 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:36.906978 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:36.906938 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:36.907444 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:36.906939 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:36.907444 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:36.907086 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:36.907444 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:36.907098 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:36.907444 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:36.907173 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:36.907444 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:36.907244 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:38.652193 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:38.652104 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:38.652193 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:38.652167 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:38.652752 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:38.652199 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrqg\" (UniqueName: \"kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg\") pod \"network-check-target-jtn82\" (UID: \"adc643af-b310-4c8a-a4b5-d96749bf4188\") " pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:38.652752 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:38.652260 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:38.652752 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:38.652306 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 21:00:38.652752 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:38.652320 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 21:00:38.652752 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:38.652329 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs podName:819c1153-e1db-4114-8e57-eb79d24d8db9 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:54.652308381 +0000 UTC m=+34.263061744 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs") pod "network-metrics-daemon-ckmm7" (UID: "819c1153-e1db-4114-8e57-eb79d24d8db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:38.652752 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:38.652333 2568 projected.go:196] Error preparing data for projected volume kube-api-access-qgrqg for pod openshift-network-diagnostics/network-check-target-jtn82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:38.652752 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:38.652331 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:38.652752 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:38.652368 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg podName:adc643af-b310-4c8a-a4b5-d96749bf4188 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:54.652358359 +0000 UTC m=+34.263111720 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgrqg" (UniqueName: "kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg") pod "network-check-target-jtn82" (UID: "adc643af-b310-4c8a-a4b5-d96749bf4188") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:38.652752 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:38.652389 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret podName:681e613d-9c45-4255-be57-f765bec9fb14 nodeName:}" failed. No retries permitted until 2026-07-21 21:00:54.652375458 +0000 UTC m=+34.263128822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret") pod "global-pull-secret-syncer-pchv7" (UID: "681e613d-9c45-4255-be57-f765bec9fb14") : object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:38.906068 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:38.905990 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:38.906225 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:38.906001 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:38.906225 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:38.906094 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:38.906225 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:38.906113 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:38.906225 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:38.906208 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:38.906437 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:38.906282 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:40.909347 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:40.909314 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:40.909864 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:40.909313 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:40.909864 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:40.909485 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:40.909864 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:40.909317 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:40.909864 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:40.909542 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:40.909864 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:40.909643 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:42.906824 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:42.906795 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:42.906824 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:42.906813 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:42.907212 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:42.906813 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:42.907212 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:42.906896 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:42.907212 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:42.906969 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:42.907212 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:42.907047 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:44.015488 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.015181 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cghfw" event={"ID":"7f05adfe-0e27-4ea2-8e61-4e03c16fba01","Type":"ContainerStarted","Data":"33a8393f353ceafcc456908321ab255c34a7cbeb3a9114745c695c44d8bac7cf"} Jul 21 21:00:44.016732 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.016710 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" event={"ID":"0ad7a23a-99f7-4e57-8ebd-5f1513ffae20","Type":"ContainerStarted","Data":"fd22999a9564d31b772b57c3480622a3940f19760436e08f009a318a17896d98"} Jul 21 21:00:44.017999 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.017976 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" event={"ID":"35e82a47-c026-420d-9e9a-fb3e524ff0b6","Type":"ContainerStarted","Data":"69c33d5b4611b4fdd9674396a9cde281f10cf681922d1f76e48b78cb1e142816"} Jul 21 21:00:44.020213 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.020197 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:00:44.020529 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.020513 2568 generic.go:356] "Generic (PLEG): container finished" podID="4ef7d53f-cca5-4dc9-9c36-d064660f3f99" containerID="0eacddebd53de6174f84d10dfef58c0ebda04d9d8999a427ec2f44eeb3f1ae05" exitCode=1 Jul 21 21:00:44.020581 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.020567 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" event={"ID":"4ef7d53f-cca5-4dc9-9c36-d064660f3f99","Type":"ContainerStarted","Data":"61409a4540dc24bc502811412c135c179c0ffd8e4389055020b861040da55382"} Jul 21 21:00:44.020659 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.020635 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" event={"ID":"4ef7d53f-cca5-4dc9-9c36-d064660f3f99","Type":"ContainerStarted","Data":"771f788467cf331053818f59f646ce64d0a30bff0c40a4446f86f14b919533c0"} Jul 21 21:00:44.020696 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.020682 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" event={"ID":"4ef7d53f-cca5-4dc9-9c36-d064660f3f99","Type":"ContainerStarted","Data":"d636c3b4619a2de8b5aa76cf0b57672e10fc931e92c5e07a5694457b2ca80222"} Jul 21 21:00:44.020734 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.020713 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" event={"ID":"4ef7d53f-cca5-4dc9-9c36-d064660f3f99","Type":"ContainerDied","Data":"0eacddebd53de6174f84d10dfef58c0ebda04d9d8999a427ec2f44eeb3f1ae05"} Jul 21 21:00:44.020768 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.020740 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" event={"ID":"4ef7d53f-cca5-4dc9-9c36-d064660f3f99","Type":"ContainerStarted","Data":"35ebab70e020ee62dd2df7f7319ebc0d1e3d83a48e10b4681b4d01ab395d26d3"} Jul 21 21:00:44.021797 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.021778 2568 generic.go:356] "Generic (PLEG): container finished" podID="9bb98f87-4d9c-4654-9e10-e52bfb05591f" containerID="9a355cdbea83e9f5028e892485dc409e83d0faefba4a611f8e3cae5199f44c07" exitCode=0 Jul 21 21:00:44.021879 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.021858 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqk2d" event={"ID":"9bb98f87-4d9c-4654-9e10-e52bfb05591f","Type":"ContainerDied","Data":"9a355cdbea83e9f5028e892485dc409e83d0faefba4a611f8e3cae5199f44c07"} Jul 21 21:00:44.023149 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.023131 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pmwz" event={"ID":"65f51fb4-b664-45da-81ab-4a0d20506ad2","Type":"ContainerStarted","Data":"45f81788f7999dec7474909760736590d5af25f0008416c3bc0fc6332dee99aa"} Jul 21 21:00:44.024607 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.024571 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-94p6s" event={"ID":"cd4e8142-0742-404d-a5d1-f21f20672947","Type":"ContainerStarted","Data":"ff1fbe8a8426503029b307d7a1a8ef7ae5dbc0c7904326a9adcf60dbbed0ed44"} Jul 21 21:00:44.025919 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.025897 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vxtql" event={"ID":"a816a2e0-0ddf-42e2-ad1b-54b69582b666","Type":"ContainerStarted","Data":"3d7d03cf63f4b4a13c817134d44c9bf8785272d16065bd5590ff00fd23860c64"} Jul 21 21:00:44.030662 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.030626 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cghfw" podStartSLOduration=2.960840232 podStartE2EDuration="23.030615385s" totalImagesPullingTime="20.069775153s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:44.030443437 +0000 UTC m=+23.641196820" watchObservedRunningTime="2026-07-21 21:00:44.030615385 +0000 UTC m=+23.641368766" Jul 21 21:00:44.030944 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.030925 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" podStartSLOduration=22.030920176 podStartE2EDuration="22.030920176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:27.00081137 +0000 UTC m=+6.611564780" watchObservedRunningTime="2026-07-21 21:00:44.030920176 +0000 UTC m=+23.641673560" Jul 21 21:00:44.046106 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.046073 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2pmwz" podStartSLOduration=2.9397130110000003 podStartE2EDuration="23.046061914s" totalImagesPullingTime="20.106348903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:44.04563092 +0000 UTC m=+23.656384304" watchObservedRunningTime="2026-07-21 21:00:44.046061914 +0000 UTC m=+23.656815297" Jul 21 21:00:44.069504 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.069472 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-94p6s" podStartSLOduration=2.971221079 podStartE2EDuration="23.069464675s" totalImagesPullingTime="20.098243596s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:44.068696427 +0000 UTC m=+23.679449836" watchObservedRunningTime="2026-07-21 21:00:44.069464675 +0000 UTC m=+23.680218059" Jul 21 21:00:44.091488 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.091458 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4b7nq" podStartSLOduration=3.013666331 podStartE2EDuration="23.091446054s" totalImagesPullingTime="20.077779723s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:44.091427037 +0000 UTC m=+23.702180420" watchObservedRunningTime="2026-07-21 21:00:44.091446054 +0000 UTC m=+23.702199437" Jul 21 21:00:44.848887 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.848841 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 21:00:44.882244 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.882143 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T21:00:44.848862771Z","UUID":"fd0cbf97-0121-45c4-865b-c5e877df24b5","Handler":null,"Name":"","Endpoint":""} Jul 21 21:00:44.883594 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.883570 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 21:00:44.883680 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.883625 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 21:00:44.905975 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.905948 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:44.906074 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.905948 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:44.906074 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:44.906057 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:44.906181 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:44.906077 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:44.906238 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:44.906169 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:44.906300 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:44.906279 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:45.029357 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:45.029274 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lsc9f" event={"ID":"f16baa81-ef27-43dc-9892-58ab92744f43","Type":"ContainerStarted","Data":"dd7d15ca76ec22a0caba41275fcb7d004992dad135ee6cb8e06717bb89472dd7"} Jul 21 21:00:45.031390 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:45.031365 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" event={"ID":"35e82a47-c026-420d-9e9a-fb3e524ff0b6","Type":"ContainerStarted","Data":"697da9810c805c8a47bed178ee875ba52f6b3f784d562396dd8b2474ef49cf0c"} Jul 21 21:00:45.034538 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:45.034518 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:00:45.034938 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:45.034911 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" event={"ID":"4ef7d53f-cca5-4dc9-9c36-d064660f3f99","Type":"ContainerStarted","Data":"d3c88fec0a0e0bf3f2f8f7cb87f45e8c7c439e68aa3de0f8a3eda3f1b0943dab"} Jul 21 21:00:45.045973 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:45.045933 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vxtql" podStartSLOduration=8.783768538 podStartE2EDuration="24.045918141s" totalImagesPullingTime="15.262149603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:44.150664503 +0000 UTC m=+23.761417889" watchObservedRunningTime="2026-07-21 21:00:45.045918141 +0000 UTC m=+24.656671507" Jul 21 21:00:45.046660 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:45.046619 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lsc9f" podStartSLOduration=4.00755225 podStartE2EDuration="24.046606101s" totalImagesPullingTime="20.039053851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:45.045824298 +0000 UTC m=+24.656577682" watchObservedRunningTime="2026-07-21 21:00:45.046606101 +0000 UTC m=+24.657359485" Jul 21 21:00:46.906233 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:46.906055 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:46.906723 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:46.906055 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:46.906723 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:46.906066 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:46.906723 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:46.906549 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:46.906873 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:46.906789 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:46.906980 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:46.906920 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:47.042949 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:47.042920 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:00:47.043348 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:47.043323 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" event={"ID":"4ef7d53f-cca5-4dc9-9c36-d064660f3f99","Type":"ContainerStarted","Data":"0ee7f41e60a503e7b214cdf53fd3b2722878cbf104326a34dec8ded21cdb3d3e"} Jul 21 21:00:47.045700 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:47.045669 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" event={"ID":"35e82a47-c026-420d-9e9a-fb3e524ff0b6","Type":"ContainerStarted","Data":"7bc39e46ff1a3c3311716983682d7d6d23df6fdfe80c6c4cd6d9a1ebee85f768"} Jul 21 21:00:47.064621 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:47.064557 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c6sg9" podStartSLOduration=3.282410435 podStartE2EDuration="26.064543358s" totalImagesPullingTime="22.782132923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:47.064214984 +0000 UTC m=+26.674968367" watchObservedRunningTime="2026-07-21 21:00:47.064543358 +0000 UTC m=+26.675296741" Jul 21 21:00:48.906197 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:48.906156 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:48.906197 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:48.906187 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:48.906821 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:48.906163 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:48.906821 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:48.906273 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:48.906821 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:48.906334 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:48.906821 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:48.906400 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:48.944730 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:48.944704 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:48.945273 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:48.945254 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:49.053728 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:49.053707 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:00:49.054112 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:49.054087 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" event={"ID":"4ef7d53f-cca5-4dc9-9c36-d064660f3f99","Type":"ContainerStarted","Data":"1ba5b683bab8a82b059ac2c40a77af6a44b51c002917adfe509e36970ad84610"} Jul 21 21:00:49.054203 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:49.054189 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:49.054630 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:49.054614 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vxtql" Jul 21 21:00:50.057704 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:50.057509 2568 generic.go:356] "Generic (PLEG): container finished" podID="9bb98f87-4d9c-4654-9e10-e52bfb05591f" containerID="247be76d9b5927f10cf8faa5f56c0f25bb50ae026230f93c3494e6a2705ba47b" exitCode=0 Jul 21 21:00:50.058769 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:50.057614 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqk2d" event={"ID":"9bb98f87-4d9c-4654-9e10-e52bfb05591f","Type":"ContainerDied","Data":"247be76d9b5927f10cf8faa5f56c0f25bb50ae026230f93c3494e6a2705ba47b"} Jul 21 21:00:50.060150 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:50.059092 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:50.060150 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:50.059243 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:50.060150 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:50.059298 2568 scope.go:117] "RemoveContainer" containerID="0eacddebd53de6174f84d10dfef58c0ebda04d9d8999a427ec2f44eeb3f1ae05" Jul 21 21:00:50.060150 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:50.059614 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:50.077333 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:50.077314 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:50.077730 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:50.077713 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:00:50.909094 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:50.909067 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:50.909326 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:50.909074 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:50.909326 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:50.909160 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:50.909326 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:50.909167 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:50.909326 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:50.909224 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:50.909326 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:50.909282 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:51.064676 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:51.064655 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:00:51.065071 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:51.065015 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" event={"ID":"4ef7d53f-cca5-4dc9-9c36-d064660f3f99","Type":"ContainerStarted","Data":"ff1031f933a7c702c5ebec0b7015ec72c9828b7cff7f14b7ccd3f5ec6b572158"} Jul 21 21:00:51.104598 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:51.104530 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" podStartSLOduration=9.930009172 podStartE2EDuration="30.104510441s" totalImagesPullingTime="20.174501269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:51.102877963 +0000 UTC m=+30.713631346" watchObservedRunningTime="2026-07-21 21:00:51.104510441 +0000 UTC m=+30.715263825" Jul 21 21:00:51.215677 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:51.215648 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pchv7"] Jul 21 21:00:51.215815 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:51.215802 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:51.215924 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:51.215906 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:51.219633 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:51.219612 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ckmm7"] Jul 21 21:00:51.219741 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:51.219715 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:51.219835 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:51.219818 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:51.222181 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:51.222158 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jtn82"] Jul 21 21:00:51.222252 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:51.222246 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:51.222346 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:51.222321 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:52.069538 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:52.069497 2568 generic.go:356] "Generic (PLEG): container finished" podID="9bb98f87-4d9c-4654-9e10-e52bfb05591f" containerID="16310d1e9fac298ee8165217a2f4c5327779c5eeb50daf25db54934e61f8684c" exitCode=0 Jul 21 21:00:52.069907 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:52.069533 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqk2d" event={"ID":"9bb98f87-4d9c-4654-9e10-e52bfb05591f","Type":"ContainerDied","Data":"16310d1e9fac298ee8165217a2f4c5327779c5eeb50daf25db54934e61f8684c"} Jul 21 21:00:52.906401 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:52.906374 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:52.906568 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:52.906375 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:52.906568 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:52.906474 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:52.906691 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:52.906375 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:52.906691 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:52.906576 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:52.906767 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:52.906702 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:54.677491 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:54.677274 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrqg\" (UniqueName: \"kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg\") pod \"network-check-target-jtn82\" (UID: \"adc643af-b310-4c8a-a4b5-d96749bf4188\") " pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:54.677491 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:54.677428 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 21:00:54.677491 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:54.677446 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 21:00:54.677491 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:54.677455 2568 projected.go:196] Error preparing data for projected volume kube-api-access-qgrqg for pod openshift-network-diagnostics/network-check-target-jtn82: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:54.677491 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:54.677481 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:54.678151 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:54.677500 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg podName:adc643af-b310-4c8a-a4b5-d96749bf4188 nodeName:}" failed. No retries permitted until 2026-07-21 21:01:26.67748587 +0000 UTC m=+66.288239254 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qgrqg" (UniqueName: "kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg") pod "network-check-target-jtn82" (UID: "adc643af-b310-4c8a-a4b5-d96749bf4188") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 21:00:54.678151 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:54.677539 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:54.678151 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:54.677570 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:54.678151 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:54.677638 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:54.678151 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:54.677644 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs podName:819c1153-e1db-4114-8e57-eb79d24d8db9 nodeName:}" failed. No retries permitted until 2026-07-21 21:01:26.677629964 +0000 UTC m=+66.288383326 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs") pod "network-metrics-daemon-ckmm7" (UID: "819c1153-e1db-4114-8e57-eb79d24d8db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 21:00:54.678151 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:54.677675 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret podName:681e613d-9c45-4255-be57-f765bec9fb14 nodeName:}" failed. No retries permitted until 2026-07-21 21:01:26.677662688 +0000 UTC m=+66.288416050 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret") pod "global-pull-secret-syncer-pchv7" (UID: "681e613d-9c45-4255-be57-f765bec9fb14") : object "kube-system"/"original-pull-secret" not registered Jul 21 21:00:54.906777 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:54.906747 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:54.906912 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:54.906748 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:54.906912 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:54.906877 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:54.906912 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:54.906894 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:54.907025 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:54.906950 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:54.907069 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:54.907036 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:55.077559 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:55.077532 2568 generic.go:356] "Generic (PLEG): container finished" podID="9bb98f87-4d9c-4654-9e10-e52bfb05591f" containerID="fe7a3fe46c58074150f14ed2673bd2cfeb9ce30dd3c7ddbb8a98352f046f8c2f" exitCode=0 Jul 21 21:00:55.077700 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:55.077606 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqk2d" event={"ID":"9bb98f87-4d9c-4654-9e10-e52bfb05591f","Type":"ContainerDied","Data":"fe7a3fe46c58074150f14ed2673bd2cfeb9ce30dd3c7ddbb8a98352f046f8c2f"} Jul 21 21:00:56.906628 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:56.906580 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:56.907138 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:56.906579 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:56.907138 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:56.906727 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:56.907138 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:56.906723 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckmm7" podUID="819c1153-e1db-4114-8e57-eb79d24d8db9" Jul 21 21:00:56.907138 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:56.906815 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pchv7" podUID="681e613d-9c45-4255-be57-f765bec9fb14" Jul 21 21:00:56.907138 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:56.906898 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jtn82" podUID="adc643af-b310-4c8a-a4b5-d96749bf4188" Jul 21 21:00:57.166479 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.166448 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeReady" Jul 21 21:00:57.166668 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.166581 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 21:00:57.229915 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.229886 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8b4459dcb-z5rpn"] Jul 21 21:00:57.233292 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.233268 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.236059 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.236040 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-58h5p\"" Jul 21 21:00:57.237160 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.237140 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 21:00:57.237441 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.237423 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 21:00:57.237998 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.237974 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 21:00:57.247708 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.247687 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qflql"] Jul 21 21:00:57.248845 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.248825 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 21:00:57.252017 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.251998 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.252282 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.252259 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7xpvd"] Jul 21 21:00:57.255988 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.255969 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7xpvd" Jul 21 21:00:57.256442 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.256410 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 21:00:57.256671 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.256655 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 21:00:57.256850 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.256835 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wp8nm\"" Jul 21 21:00:57.257672 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.257654 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8b4459dcb-z5rpn"] Jul 21 21:00:57.261682 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.261662 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 21:00:57.261786 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.261711 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bv9cb\"" Jul 21 21:00:57.261786 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.261734 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 21:00:57.261895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.261772 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 21:00:57.267116 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.267057 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qflql"] Jul 21 21:00:57.267947 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.267820 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7xpvd"] Jul 21 21:00:57.397678 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.397653 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7561c51-58dc-4d33-89d8-974e4cc6601a-cert\") pod \"ingress-canary-7xpvd\" (UID: \"b7561c51-58dc-4d33-89d8-974e4cc6601a\") " pod="openshift-ingress-canary/ingress-canary-7xpvd" Jul 21 21:00:57.397678 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.397683 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.397886 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.397711 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz2x4\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-kube-api-access-vz2x4\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.397886 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.397735 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.397886 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.397762 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-bound-sa-token\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.397886 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.397780 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-trusted-ca\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.397886 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.397801 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-installation-pull-secrets\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.397886 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.397820 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-ca-trust-extracted\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.397886 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.397837 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l925\" (UniqueName: \"kubernetes.io/projected/b7561c51-58dc-4d33-89d8-974e4cc6601a-kube-api-access-7l925\") pod \"ingress-canary-7xpvd\" (UID: \"b7561c51-58dc-4d33-89d8-974e4cc6601a\") " pod="openshift-ingress-canary/ingress-canary-7xpvd" Jul 21 21:00:57.398189 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.397943 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-config-volume\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.398189 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.397982 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-image-registry-private-configuration\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.398189 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.398012 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-tmp-dir\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.398189 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.398038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzvzz\" (UniqueName: \"kubernetes.io/projected/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-kube-api-access-kzvzz\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.398189 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.398067 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-certificates\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.498629 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.498534 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-trusted-ca\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.498629 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.498575 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-installation-pull-secrets\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.498629 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.498620 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-ca-trust-extracted\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.498885 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.498639 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l925\" (UniqueName: \"kubernetes.io/projected/b7561c51-58dc-4d33-89d8-974e4cc6601a-kube-api-access-7l925\") pod \"ingress-canary-7xpvd\" (UID: \"b7561c51-58dc-4d33-89d8-974e4cc6601a\") " pod="openshift-ingress-canary/ingress-canary-7xpvd" Jul 21 21:00:57.498885 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.498675 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-config-volume\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.499146 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499099 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-image-registry-private-configuration\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.499264 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499157 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-tmp-dir\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.499264 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzvzz\" (UniqueName: \"kubernetes.io/projected/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-kube-api-access-kzvzz\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.499264 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499215 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-certificates\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.499264 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499265 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7561c51-58dc-4d33-89d8-974e4cc6601a-cert\") pod \"ingress-canary-7xpvd\" (UID: \"b7561c51-58dc-4d33-89d8-974e4cc6601a\") " pod="openshift-ingress-canary/ingress-canary-7xpvd" Jul 21 21:00:57.499471 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499280 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.499471 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499298 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-config-volume\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.499471 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499306 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz2x4\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-kube-api-access-vz2x4\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.499471 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499369 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.499471 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499412 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-bound-sa-token\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.499728 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499560 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-tmp-dir\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.499728 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499106 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-ca-trust-extracted\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.499728 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499685 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-trusted-ca\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.499728 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:57.499692 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 21:00:57.499728 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:57.499705 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 21:00:57.499728 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:57.499729 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8b4459dcb-z5rpn: secret "image-registry-tls" not found Jul 21 21:00:57.499976 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:57.499774 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls podName:ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b nodeName:}" failed. No retries permitted until 2026-07-21 21:00:57.99975289 +0000 UTC m=+37.610506258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls") pod "dns-default-qflql" (UID: "ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b") : secret "dns-default-metrics-tls" not found Jul 21 21:00:57.499976 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:57.499809 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls podName:32d3be26-c94d-47e2-90f0-93fa49ce1c8f nodeName:}" failed. No retries permitted until 2026-07-21 21:00:57.999781354 +0000 UTC m=+37.610534717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls") pod "image-registry-8b4459dcb-z5rpn" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f") : secret "image-registry-tls" not found Jul 21 21:00:57.499976 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.499931 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-certificates\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.503599 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.503558 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-installation-pull-secrets\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.503719 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.503564 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-image-registry-private-configuration\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.503719 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.503702 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7561c51-58dc-4d33-89d8-974e4cc6601a-cert\") pod \"ingress-canary-7xpvd\" (UID: \"b7561c51-58dc-4d33-89d8-974e4cc6601a\") " pod="openshift-ingress-canary/ingress-canary-7xpvd" Jul 21 21:00:57.509894 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.509855 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l925\" (UniqueName: \"kubernetes.io/projected/b7561c51-58dc-4d33-89d8-974e4cc6601a-kube-api-access-7l925\") pod \"ingress-canary-7xpvd\" (UID: \"b7561c51-58dc-4d33-89d8-974e4cc6601a\") " pod="openshift-ingress-canary/ingress-canary-7xpvd" Jul 21 21:00:57.510671 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.510651 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz2x4\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-kube-api-access-vz2x4\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.512797 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.512773 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-bound-sa-token\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:57.514874 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.514853 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzvzz\" (UniqueName: \"kubernetes.io/projected/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-kube-api-access-kzvzz\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:57.571875 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.571848 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7xpvd" Jul 21 21:00:57.742062 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:57.741841 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7xpvd"] Jul 21 21:00:57.746185 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:00:57.746141 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7561c51_58dc_4d33_89d8_974e4cc6601a.slice/crio-52b5e8238ea1759a8b22ea693d75feda527bb58a6ca32a64730ee2e3184ae4c7 WatchSource:0}: Error finding container 52b5e8238ea1759a8b22ea693d75feda527bb58a6ca32a64730ee2e3184ae4c7: Status 404 returned error can't find the container with id 52b5e8238ea1759a8b22ea693d75feda527bb58a6ca32a64730ee2e3184ae4c7 Jul 21 21:00:58.003700 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.003620 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:58.004202 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.003723 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:58.004202 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:58.003786 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 21:00:58.004202 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:58.003831 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 21:00:58.004202 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:58.003841 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8b4459dcb-z5rpn: secret "image-registry-tls" not found Jul 21 21:00:58.004202 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:58.003862 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls podName:ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b nodeName:}" failed. No retries permitted until 2026-07-21 21:00:59.003843664 +0000 UTC m=+38.614597031 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls") pod "dns-default-qflql" (UID: "ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b") : secret "dns-default-metrics-tls" not found Jul 21 21:00:58.004202 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:58.003883 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls podName:32d3be26-c94d-47e2-90f0-93fa49ce1c8f nodeName:}" failed. No retries permitted until 2026-07-21 21:00:59.003868771 +0000 UTC m=+38.614622343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls") pod "image-registry-8b4459dcb-z5rpn" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f") : secret "image-registry-tls" not found Jul 21 21:00:58.085793 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.085755 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7xpvd" event={"ID":"b7561c51-58dc-4d33-89d8-974e4cc6601a","Type":"ContainerStarted","Data":"52b5e8238ea1759a8b22ea693d75feda527bb58a6ca32a64730ee2e3184ae4c7"} Jul 21 21:00:58.911699 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.911666 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:00:58.911699 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.911684 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:00:58.911917 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.911666 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:00:58.920981 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.920310 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-slzpm\"" Jul 21 21:00:58.922822 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.922800 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 21:00:58.923578 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.923556 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cvwws\"" Jul 21 21:00:58.923824 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.923807 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 21:00:58.924307 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.924285 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 21:00:58.932778 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:58.932756 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 21:00:59.011397 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:59.011361 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:00:59.011820 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:00:59.011424 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:00:59.011820 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:59.011491 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 21:00:59.011820 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:59.011510 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8b4459dcb-z5rpn: secret "image-registry-tls" not found Jul 21 21:00:59.011820 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:59.011569 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls podName:32d3be26-c94d-47e2-90f0-93fa49ce1c8f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:01.011548708 +0000 UTC m=+40.622302088 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls") pod "image-registry-8b4459dcb-z5rpn" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f") : secret "image-registry-tls" not found Jul 21 21:00:59.011820 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:59.011572 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 21:00:59.011820 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:00:59.011644 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls podName:ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b nodeName:}" failed. No retries permitted until 2026-07-21 21:01:01.011628701 +0000 UTC m=+40.622382066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls") pod "dns-default-qflql" (UID: "ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b") : secret "dns-default-metrics-tls" not found Jul 21 21:01:00.092897 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:00.092865 2568 generic.go:356] "Generic (PLEG): container finished" podID="9bb98f87-4d9c-4654-9e10-e52bfb05591f" containerID="34b96d6826fa040837aa2e0fafb18a89ebd695d5022e0e11dbfa0cb4c6fe04a5" exitCode=0 Jul 21 21:01:00.093404 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:00.092912 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqk2d" event={"ID":"9bb98f87-4d9c-4654-9e10-e52bfb05591f","Type":"ContainerDied","Data":"34b96d6826fa040837aa2e0fafb18a89ebd695d5022e0e11dbfa0cb4c6fe04a5"} Jul 21 21:01:01.027086 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:01.027003 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:01:01.027086 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:01.027061 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:01:01.027297 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:01.027147 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 21:01:01.027297 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:01.027159 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8b4459dcb-z5rpn: secret "image-registry-tls" not found Jul 21 21:01:01.027297 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:01.027166 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 21:01:01.027297 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:01.027218 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls podName:32d3be26-c94d-47e2-90f0-93fa49ce1c8f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:05.027201434 +0000 UTC m=+44.637954816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls") pod "image-registry-8b4459dcb-z5rpn" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f") : secret "image-registry-tls" not found Jul 21 21:01:01.027297 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:01.027236 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls podName:ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b nodeName:}" failed. No retries permitted until 2026-07-21 21:01:05.027227757 +0000 UTC m=+44.637981128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls") pod "dns-default-qflql" (UID: "ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b") : secret "dns-default-metrics-tls" not found Jul 21 21:01:01.097904 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:01.097874 2568 generic.go:356] "Generic (PLEG): container finished" podID="9bb98f87-4d9c-4654-9e10-e52bfb05591f" containerID="3cd70e0964969031d1692cb832e762d36d666f2cea680b02ee5942e902e23e6b" exitCode=0 Jul 21 21:01:01.098255 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:01.097942 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqk2d" event={"ID":"9bb98f87-4d9c-4654-9e10-e52bfb05591f","Type":"ContainerDied","Data":"3cd70e0964969031d1692cb832e762d36d666f2cea680b02ee5942e902e23e6b"} Jul 21 21:01:02.103025 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:02.102986 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zqk2d" event={"ID":"9bb98f87-4d9c-4654-9e10-e52bfb05591f","Type":"ContainerStarted","Data":"8bd7250a5a3623b6b9c041a49a65d8a93e68b7970e742884abdbb2a49d04ab77"} Jul 21 21:01:02.133782 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:02.133741 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zqk2d" podStartSLOduration=8.499364161 podStartE2EDuration="41.1337246s" totalImagesPullingTime="32.634360439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:01:02.131697827 +0000 UTC m=+41.742451211" watchObservedRunningTime="2026-07-21 21:01:02.1337246 +0000 UTC m=+41.744477983" Jul 21 21:01:03.107296 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:03.107260 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7xpvd" event={"ID":"b7561c51-58dc-4d33-89d8-974e4cc6601a","Type":"ContainerStarted","Data":"f5105c1843141621cf283b027eedba2ac0b3bb978fcf4e62fdff23ef6a07ed5f"} Jul 21 21:01:03.125578 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:03.125531 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7xpvd" podStartSLOduration=1.494355511 podStartE2EDuration="6.125517132s" totalImagesPullingTime="4.631161621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:01:03.124318478 +0000 UTC m=+42.735071861" watchObservedRunningTime="2026-07-21 21:01:03.125517132 +0000 UTC m=+42.736270516" Jul 21 21:01:05.053228 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:05.053192 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:01:05.053612 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:05.053240 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:01:05.053612 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:05.053338 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 21:01:05.053612 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:05.053345 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 21:01:05.053612 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:05.053362 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8b4459dcb-z5rpn: secret "image-registry-tls" not found Jul 21 21:01:05.053612 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:05.053387 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls podName:ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b nodeName:}" failed. No retries permitted until 2026-07-21 21:01:13.053373946 +0000 UTC m=+52.664127308 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls") pod "dns-default-qflql" (UID: "ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b") : secret "dns-default-metrics-tls" not found Jul 21 21:01:05.053612 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:05.053406 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls podName:32d3be26-c94d-47e2-90f0-93fa49ce1c8f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:13.053392286 +0000 UTC m=+52.664145648 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls") pod "image-registry-8b4459dcb-z5rpn" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f") : secret "image-registry-tls" not found Jul 21 21:01:13.106434 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:13.106390 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:01:13.106434 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:13.106442 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:01:13.106914 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:13.106547 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 21:01:13.106914 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:13.106551 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 21:01:13.106914 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:13.106570 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8b4459dcb-z5rpn: secret "image-registry-tls" not found Jul 21 21:01:13.106914 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:13.106623 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls podName:ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b nodeName:}" failed. No retries permitted until 2026-07-21 21:01:29.106608429 +0000 UTC m=+68.717361791 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls") pod "dns-default-qflql" (UID: "ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b") : secret "dns-default-metrics-tls" not found Jul 21 21:01:13.106914 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:13.106637 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls podName:32d3be26-c94d-47e2-90f0-93fa49ce1c8f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:29.106630998 +0000 UTC m=+68.717384359 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls") pod "image-registry-8b4459dcb-z5rpn" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f") : secret "image-registry-tls" not found Jul 21 21:01:22.080867 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:22.080839 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77bjb" Jul 21 21:01:26.702477 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.702445 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:01:26.702977 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.702487 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:01:26.702977 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.702516 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrqg\" (UniqueName: \"kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg\") pod \"network-check-target-jtn82\" (UID: \"adc643af-b310-4c8a-a4b5-d96749bf4188\") " pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:01:26.705459 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.705441 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 21:01:26.705635 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.705623 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 21:01:26.705705 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.705687 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 21:01:26.712726 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:26.712708 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 21 21:01:26.712802 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:26.712789 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs podName:819c1153-e1db-4114-8e57-eb79d24d8db9 nodeName:}" failed. No retries permitted until 2026-07-21 21:02:30.712769271 +0000 UTC m=+130.323522634 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs") pod "network-metrics-daemon-ckmm7" (UID: "819c1153-e1db-4114-8e57-eb79d24d8db9") : secret "metrics-daemon-secret" not found Jul 21 21:01:26.715165 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.715152 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 21:01:26.716129 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.716110 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/681e613d-9c45-4255-be57-f765bec9fb14-original-pull-secret\") pod \"global-pull-secret-syncer-pchv7\" (UID: \"681e613d-9c45-4255-be57-f765bec9fb14\") " pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:01:26.725523 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.725497 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgrqg\" (UniqueName: \"kubernetes.io/projected/adc643af-b310-4c8a-a4b5-d96749bf4188-kube-api-access-qgrqg\") pod \"network-check-target-jtn82\" (UID: \"adc643af-b310-4c8a-a4b5-d96749bf4188\") " pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:01:26.827145 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.827124 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-slzpm\"" Jul 21 21:01:26.830150 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.830138 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pchv7" Jul 21 21:01:26.834932 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.834919 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:01:26.983655 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:26.983573 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pchv7"] Jul 21 21:01:26.986823 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:01:26.986794 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod681e613d_9c45_4255_be57_f765bec9fb14.slice/crio-7df66b8a7d34172890fa330ccbe53b20f19ce34bf9862a1c65416a2402f23cb7 WatchSource:0}: Error finding container 7df66b8a7d34172890fa330ccbe53b20f19ce34bf9862a1c65416a2402f23cb7: Status 404 returned error can't find the container with id 7df66b8a7d34172890fa330ccbe53b20f19ce34bf9862a1c65416a2402f23cb7 Jul 21 21:01:27.000125 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:27.000102 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jtn82"] Jul 21 21:01:27.002066 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:01:27.002041 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadc643af_b310_4c8a_a4b5_d96749bf4188.slice/crio-74e68bdf19045687434551b8e27d75fd0c428dbaf56e46d616203333b1e110a2 WatchSource:0}: Error finding container 74e68bdf19045687434551b8e27d75fd0c428dbaf56e46d616203333b1e110a2: Status 404 returned error can't find the container with id 74e68bdf19045687434551b8e27d75fd0c428dbaf56e46d616203333b1e110a2 Jul 21 21:01:27.158945 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:27.158911 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jtn82" event={"ID":"adc643af-b310-4c8a-a4b5-d96749bf4188","Type":"ContainerStarted","Data":"74e68bdf19045687434551b8e27d75fd0c428dbaf56e46d616203333b1e110a2"} Jul 21 21:01:27.159902 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:27.159882 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pchv7" event={"ID":"681e613d-9c45-4255-be57-f765bec9fb14","Type":"ContainerStarted","Data":"7df66b8a7d34172890fa330ccbe53b20f19ce34bf9862a1c65416a2402f23cb7"} Jul 21 21:01:29.119173 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:29.119135 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:01:29.119663 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:29.119186 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:01:29.119663 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:29.119327 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 21:01:29.119663 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:29.119355 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8b4459dcb-z5rpn: secret "image-registry-tls" not found Jul 21 21:01:29.119663 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:29.119423 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls podName:32d3be26-c94d-47e2-90f0-93fa49ce1c8f nodeName:}" failed. No retries permitted until 2026-07-21 21:02:01.119399351 +0000 UTC m=+100.730152725 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls") pod "image-registry-8b4459dcb-z5rpn" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f") : secret "image-registry-tls" not found Jul 21 21:01:29.119663 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:29.119330 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 21:01:29.119663 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:29.119483 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls podName:ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b nodeName:}" failed. No retries permitted until 2026-07-21 21:02:01.119466118 +0000 UTC m=+100.730219494 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls") pod "dns-default-qflql" (UID: "ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b") : secret "dns-default-metrics-tls" not found Jul 21 21:01:32.175650 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:32.175609 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jtn82" event={"ID":"adc643af-b310-4c8a-a4b5-d96749bf4188","Type":"ContainerStarted","Data":"04d50f8133cc1d62ef64f797916d4056d7377194aedd6c19ae1014948cd7d72d"} Jul 21 21:01:32.176161 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:32.175746 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:01:32.177094 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:32.177067 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pchv7" event={"ID":"681e613d-9c45-4255-be57-f765bec9fb14","Type":"ContainerStarted","Data":"7f0a8cdb5d5532a0a6c2d189c532646ef6d9e9cadd76286a5cdf095ffd70d173"} Jul 21 21:01:32.209061 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:32.209023 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jtn82" podStartSLOduration=66.445638186 podStartE2EDuration="1m11.209010601s" totalImagesPullingTime="4.763372415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:01:32.193371599 +0000 UTC m=+71.804124984" watchObservedRunningTime="2026-07-21 21:01:32.209010601 +0000 UTC m=+71.819763984" Jul 21 21:01:33.708649 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.708576 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pchv7" podStartSLOduration=67.925280427 podStartE2EDuration="1m12.708558872s" totalImagesPullingTime="4.783278445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:01:32.210266392 +0000 UTC m=+71.821019775" watchObservedRunningTime="2026-07-21 21:01:33.708558872 +0000 UTC m=+73.319312263" Jul 21 21:01:33.709034 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.709017 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ls9lw"] Jul 21 21:01:33.711777 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.711759 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.715924 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.715897 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 21 21:01:33.716804 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.716780 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r"] Jul 21 21:01:33.716979 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.716960 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 21 21:01:33.717538 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.717516 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-qcwzj\"" Jul 21 21:01:33.717667 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.717544 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 21 21:01:33.718110 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.717961 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 21 21:01:33.719965 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.719950 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:33.721559 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.721540 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 21 21:01:33.722318 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.722304 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 21 21:01:33.722375 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.722307 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 21 21:01:33.722424 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.722395 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 21 21:01:33.722470 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.722421 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vd4s7\"" Jul 21 21:01:33.728813 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.728793 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ls9lw"] Jul 21 21:01:33.732068 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.732046 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r"] Jul 21 21:01:33.753546 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.753526 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xfsj\" (UniqueName: \"kubernetes.io/projected/9148cdbf-145d-4850-8941-686d1a7db2cd-kube-api-access-7xfsj\") pod \"cluster-samples-operator-79b7c55f87-2l29r\" (UID: \"9148cdbf-145d-4850-8941-686d1a7db2cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:33.753653 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.753555 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/604ec39e-5292-4177-a045-5f639fc3d042-config\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.753653 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.753571 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftgcr\" (UniqueName: \"kubernetes.io/projected/604ec39e-5292-4177-a045-5f639fc3d042-kube-api-access-ftgcr\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.753653 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.753612 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-2l29r\" (UID: \"9148cdbf-145d-4850-8941-686d1a7db2cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:33.753756 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.753687 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/604ec39e-5292-4177-a045-5f639fc3d042-serving-cert\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.753756 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.753738 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/604ec39e-5292-4177-a045-5f639fc3d042-trusted-ca\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.854034 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.854005 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/604ec39e-5292-4177-a045-5f639fc3d042-trusted-ca\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.854153 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.854058 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xfsj\" (UniqueName: \"kubernetes.io/projected/9148cdbf-145d-4850-8941-686d1a7db2cd-kube-api-access-7xfsj\") pod \"cluster-samples-operator-79b7c55f87-2l29r\" (UID: \"9148cdbf-145d-4850-8941-686d1a7db2cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:33.854153 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.854080 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/604ec39e-5292-4177-a045-5f639fc3d042-config\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.854153 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.854095 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftgcr\" (UniqueName: \"kubernetes.io/projected/604ec39e-5292-4177-a045-5f639fc3d042-kube-api-access-ftgcr\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.854153 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.854112 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-2l29r\" (UID: \"9148cdbf-145d-4850-8941-686d1a7db2cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:33.854153 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.854138 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/604ec39e-5292-4177-a045-5f639fc3d042-serving-cert\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.854347 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:33.854259 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 21:01:33.854347 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:33.854335 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls podName:9148cdbf-145d-4850-8941-686d1a7db2cd nodeName:}" failed. No retries permitted until 2026-07-21 21:01:34.354314788 +0000 UTC m=+73.965068151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-2l29r" (UID: "9148cdbf-145d-4850-8941-686d1a7db2cd") : secret "samples-operator-tls" not found Jul 21 21:01:33.854819 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.854745 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/604ec39e-5292-4177-a045-5f639fc3d042-config\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.855005 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.854890 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/604ec39e-5292-4177-a045-5f639fc3d042-trusted-ca\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.856446 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.856427 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/604ec39e-5292-4177-a045-5f639fc3d042-serving-cert\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.873044 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.873020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftgcr\" (UniqueName: \"kubernetes.io/projected/604ec39e-5292-4177-a045-5f639fc3d042-kube-api-access-ftgcr\") pod \"console-operator-565b7b569d-ls9lw\" (UID: \"604ec39e-5292-4177-a045-5f639fc3d042\") " pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:33.873719 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:33.873703 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xfsj\" (UniqueName: \"kubernetes.io/projected/9148cdbf-145d-4850-8941-686d1a7db2cd-kube-api-access-7xfsj\") pod \"cluster-samples-operator-79b7c55f87-2l29r\" (UID: \"9148cdbf-145d-4850-8941-686d1a7db2cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:34.021359 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:34.021284 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:34.142972 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:34.142941 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ls9lw"] Jul 21 21:01:34.146317 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:01:34.146293 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod604ec39e_5292_4177_a045_5f639fc3d042.slice/crio-6e70075bbe3c3b94d21d8439331cb65f48d26d298291eaf25c5113551a03239a WatchSource:0}: Error finding container 6e70075bbe3c3b94d21d8439331cb65f48d26d298291eaf25c5113551a03239a: Status 404 returned error can't find the container with id 6e70075bbe3c3b94d21d8439331cb65f48d26d298291eaf25c5113551a03239a Jul 21 21:01:34.183061 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:34.183032 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" event={"ID":"604ec39e-5292-4177-a045-5f639fc3d042","Type":"ContainerStarted","Data":"6e70075bbe3c3b94d21d8439331cb65f48d26d298291eaf25c5113551a03239a"} Jul 21 21:01:34.357427 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:34.357389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-2l29r\" (UID: \"9148cdbf-145d-4850-8941-686d1a7db2cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:34.357572 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:34.357537 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 21:01:34.357648 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:34.357637 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls podName:9148cdbf-145d-4850-8941-686d1a7db2cd nodeName:}" failed. No retries permitted until 2026-07-21 21:01:35.357619406 +0000 UTC m=+74.968372770 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-2l29r" (UID: "9148cdbf-145d-4850-8941-686d1a7db2cd") : secret "samples-operator-tls" not found Jul 21 21:01:35.363941 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:35.363904 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-2l29r\" (UID: \"9148cdbf-145d-4850-8941-686d1a7db2cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:35.364382 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:35.364039 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 21:01:35.364382 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:35.364145 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls podName:9148cdbf-145d-4850-8941-686d1a7db2cd nodeName:}" failed. No retries permitted until 2026-07-21 21:01:37.364121252 +0000 UTC m=+76.974874621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-2l29r" (UID: "9148cdbf-145d-4850-8941-686d1a7db2cd") : secret "samples-operator-tls" not found Jul 21 21:01:36.190501 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:36.190467 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" event={"ID":"604ec39e-5292-4177-a045-5f639fc3d042","Type":"ContainerStarted","Data":"7a418ddb1f644fbbae579b0a7f3ba9445fc837df61556506e84c88b984691482"} Jul 21 21:01:36.190713 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:36.190664 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:36.192034 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:36.192007 2568 patch_prober.go:28] interesting pod/console-operator-565b7b569d-ls9lw container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused" start-of-body= Jul 21 21:01:36.192117 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:36.192049 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" podUID="604ec39e-5292-4177-a045-5f639fc3d042" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused" Jul 21 21:01:36.207313 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:36.207249 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" podStartSLOduration=1.2548508489999999 podStartE2EDuration="3.20722698s" totalImagesPullingTime="1.952376131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:01:36.206301475 +0000 UTC m=+75.817054882" watchObservedRunningTime="2026-07-21 21:01:36.20722698 +0000 UTC m=+75.817980366" Jul 21 21:01:37.194756 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:37.194727 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/0.log" Jul 21 21:01:37.195129 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:37.194766 2568 generic.go:356] "Generic (PLEG): container finished" podID="604ec39e-5292-4177-a045-5f639fc3d042" containerID="7a418ddb1f644fbbae579b0a7f3ba9445fc837df61556506e84c88b984691482" exitCode=255 Jul 21 21:01:37.195129 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:37.194790 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" event={"ID":"604ec39e-5292-4177-a045-5f639fc3d042","Type":"ContainerDied","Data":"7a418ddb1f644fbbae579b0a7f3ba9445fc837df61556506e84c88b984691482"} Jul 21 21:01:37.195129 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:37.195029 2568 scope.go:117] "RemoveContainer" containerID="7a418ddb1f644fbbae579b0a7f3ba9445fc837df61556506e84c88b984691482" Jul 21 21:01:37.379237 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:37.379209 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-2l29r\" (UID: \"9148cdbf-145d-4850-8941-686d1a7db2cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:37.379348 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:37.379335 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 21:01:37.379404 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:37.379396 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls podName:9148cdbf-145d-4850-8941-686d1a7db2cd nodeName:}" failed. No retries permitted until 2026-07-21 21:01:41.379378427 +0000 UTC m=+80.990131806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-2l29r" (UID: "9148cdbf-145d-4850-8941-686d1a7db2cd") : secret "samples-operator-tls" not found Jul 21 21:01:38.204391 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.204361 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:01:38.204875 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.204823 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/0.log" Jul 21 21:01:38.204875 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.204854 2568 generic.go:356] "Generic (PLEG): container finished" podID="604ec39e-5292-4177-a045-5f639fc3d042" containerID="02c1bc4878b54979a7857678c033399e0170dc916f430d6ba95c4f495a5623e8" exitCode=255 Jul 21 21:01:38.205009 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.204913 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" event={"ID":"604ec39e-5292-4177-a045-5f639fc3d042","Type":"ContainerDied","Data":"02c1bc4878b54979a7857678c033399e0170dc916f430d6ba95c4f495a5623e8"} Jul 21 21:01:38.205009 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.204980 2568 scope.go:117] "RemoveContainer" containerID="7a418ddb1f644fbbae579b0a7f3ba9445fc837df61556506e84c88b984691482" Jul 21 21:01:38.205205 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.205187 2568 scope.go:117] "RemoveContainer" containerID="02c1bc4878b54979a7857678c033399e0170dc916f430d6ba95c4f495a5623e8" Jul 21 21:01:38.205370 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:38.205351 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ls9lw_openshift-console-operator(604ec39e-5292-4177-a045-5f639fc3d042)\"" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" podUID="604ec39e-5292-4177-a045-5f639fc3d042" Jul 21 21:01:38.538566 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.538490 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n2tht"] Jul 21 21:01:38.543474 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.543455 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.546346 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.546325 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 21:01:38.546566 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.546552 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 21:01:38.546961 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.546943 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 21:01:38.547053 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.546970 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 21:01:38.547053 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.546981 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vj8lm\"" Jul 21 21:01:38.554685 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.554662 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n2tht"] Jul 21 21:01:38.585872 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.585846 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10107a18-038c-41a9-847d-c96604afd79f-crio-socket\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.585956 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.585875 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxbh5\" (UniqueName: \"kubernetes.io/projected/10107a18-038c-41a9-847d-c96604afd79f-kube-api-access-gxbh5\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.585956 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.585893 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.585956 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.585917 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.585956 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.585947 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10107a18-038c-41a9-847d-c96604afd79f-data-volume\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.687214 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.687182 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10107a18-038c-41a9-847d-c96604afd79f-crio-socket\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.687386 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.687220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxbh5\" (UniqueName: \"kubernetes.io/projected/10107a18-038c-41a9-847d-c96604afd79f-kube-api-access-gxbh5\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.687386 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.687247 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.687386 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.687270 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.687386 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.687297 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10107a18-038c-41a9-847d-c96604afd79f-data-volume\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.687386 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.687315 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10107a18-038c-41a9-847d-c96604afd79f-crio-socket\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.687386 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:38.687362 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 21:01:38.687726 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:38.687393 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 21:01:38.687726 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:38.687436 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls podName:10107a18-038c-41a9-847d-c96604afd79f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:39.187415687 +0000 UTC m=+78.798169049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n2tht" (UID: "10107a18-038c-41a9-847d-c96604afd79f") : secret "insights-runtime-extractor-tls" not found Jul 21 21:01:38.687726 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:38.687459 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm podName:10107a18-038c-41a9-847d-c96604afd79f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:39.187447533 +0000 UTC m=+78.798200896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n2tht" (UID: "10107a18-038c-41a9-847d-c96604afd79f") : configmap "kube-rbac-proxy" not found Jul 21 21:01:38.687852 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.687728 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10107a18-038c-41a9-847d-c96604afd79f-data-volume\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:38.702929 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:38.702898 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxbh5\" (UniqueName: \"kubernetes.io/projected/10107a18-038c-41a9-847d-c96604afd79f-kube-api-access-gxbh5\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:39.190908 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:39.190871 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:39.190908 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:39.190910 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:39.191173 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:39.190980 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 21:01:39.191173 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:39.191021 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 21:01:39.191173 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:39.191034 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm podName:10107a18-038c-41a9-847d-c96604afd79f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:40.191019887 +0000 UTC m=+79.801773249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n2tht" (UID: "10107a18-038c-41a9-847d-c96604afd79f") : configmap "kube-rbac-proxy" not found Jul 21 21:01:39.191173 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:39.191073 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls podName:10107a18-038c-41a9-847d-c96604afd79f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:40.191059004 +0000 UTC m=+79.801812366 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n2tht" (UID: "10107a18-038c-41a9-847d-c96604afd79f") : secret "insights-runtime-extractor-tls" not found Jul 21 21:01:39.209143 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:39.209121 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:01:39.209526 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:39.209480 2568 scope.go:117] "RemoveContainer" containerID="02c1bc4878b54979a7857678c033399e0170dc916f430d6ba95c4f495a5623e8" Jul 21 21:01:39.209691 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:39.209670 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ls9lw_openshift-console-operator(604ec39e-5292-4177-a045-5f639fc3d042)\"" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" podUID="604ec39e-5292-4177-a045-5f639fc3d042" Jul 21 21:01:40.198372 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.198327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:40.198372 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.198375 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:40.198570 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:40.198443 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 21:01:40.198570 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:40.198469 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 21:01:40.198570 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:40.198507 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm podName:10107a18-038c-41a9-847d-c96604afd79f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:42.198492282 +0000 UTC m=+81.809245646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n2tht" (UID: "10107a18-038c-41a9-847d-c96604afd79f") : configmap "kube-rbac-proxy" not found Jul 21 21:01:40.198570 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:40.198522 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls podName:10107a18-038c-41a9-847d-c96604afd79f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:42.198515586 +0000 UTC m=+81.809268948 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n2tht" (UID: "10107a18-038c-41a9-847d-c96604afd79f") : secret "insights-runtime-extractor-tls" not found Jul 21 21:01:40.355088 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.355059 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-4ptst"] Jul 21 21:01:40.357841 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.357825 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-4ptst" Jul 21 21:01:40.360531 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.360511 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-pppjx\"" Jul 21 21:01:40.360842 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.360828 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 21 21:01:40.361529 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.361516 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 21 21:01:40.362570 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.362552 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 21 21:01:40.363494 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.363472 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 21 21:01:40.373058 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.373036 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-4ptst"] Jul 21 21:01:40.400504 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.400474 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cd4910b2-173b-425f-a555-0af5492f0dab-signing-cabundle\") pod \"service-ca-58598695f7-4ptst\" (UID: \"cd4910b2-173b-425f-a555-0af5492f0dab\") " pod="openshift-service-ca/service-ca-58598695f7-4ptst" Jul 21 21:01:40.400615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.400517 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cd4910b2-173b-425f-a555-0af5492f0dab-signing-key\") pod \"service-ca-58598695f7-4ptst\" (UID: \"cd4910b2-173b-425f-a555-0af5492f0dab\") " pod="openshift-service-ca/service-ca-58598695f7-4ptst" Jul 21 21:01:40.400615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.400543 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l7gs\" (UniqueName: \"kubernetes.io/projected/cd4910b2-173b-425f-a555-0af5492f0dab-kube-api-access-5l7gs\") pod \"service-ca-58598695f7-4ptst\" (UID: \"cd4910b2-173b-425f-a555-0af5492f0dab\") " pod="openshift-service-ca/service-ca-58598695f7-4ptst" Jul 21 21:01:40.501113 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.501030 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l7gs\" (UniqueName: \"kubernetes.io/projected/cd4910b2-173b-425f-a555-0af5492f0dab-kube-api-access-5l7gs\") pod \"service-ca-58598695f7-4ptst\" (UID: \"cd4910b2-173b-425f-a555-0af5492f0dab\") " pod="openshift-service-ca/service-ca-58598695f7-4ptst" Jul 21 21:01:40.501242 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.501141 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cd4910b2-173b-425f-a555-0af5492f0dab-signing-cabundle\") pod \"service-ca-58598695f7-4ptst\" (UID: \"cd4910b2-173b-425f-a555-0af5492f0dab\") " pod="openshift-service-ca/service-ca-58598695f7-4ptst" Jul 21 21:01:40.501242 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.501168 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cd4910b2-173b-425f-a555-0af5492f0dab-signing-key\") pod \"service-ca-58598695f7-4ptst\" (UID: \"cd4910b2-173b-425f-a555-0af5492f0dab\") " pod="openshift-service-ca/service-ca-58598695f7-4ptst" Jul 21 21:01:40.501930 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.501851 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cd4910b2-173b-425f-a555-0af5492f0dab-signing-cabundle\") pod \"service-ca-58598695f7-4ptst\" (UID: \"cd4910b2-173b-425f-a555-0af5492f0dab\") " pod="openshift-service-ca/service-ca-58598695f7-4ptst" Jul 21 21:01:40.503544 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.503524 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cd4910b2-173b-425f-a555-0af5492f0dab-signing-key\") pod \"service-ca-58598695f7-4ptst\" (UID: \"cd4910b2-173b-425f-a555-0af5492f0dab\") " pod="openshift-service-ca/service-ca-58598695f7-4ptst" Jul 21 21:01:40.521357 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.521315 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l7gs\" (UniqueName: \"kubernetes.io/projected/cd4910b2-173b-425f-a555-0af5492f0dab-kube-api-access-5l7gs\") pod \"service-ca-58598695f7-4ptst\" (UID: \"cd4910b2-173b-425f-a555-0af5492f0dab\") " pod="openshift-service-ca/service-ca-58598695f7-4ptst" Jul 21 21:01:40.666787 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.666757 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-4ptst" Jul 21 21:01:40.786276 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:40.786197 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-4ptst"] Jul 21 21:01:40.788871 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:01:40.788847 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd4910b2_173b_425f_a555_0af5492f0dab.slice/crio-1c65087c73021335c08e73e438f6d905d30d551c51d758bdfeba498a4f04becc WatchSource:0}: Error finding container 1c65087c73021335c08e73e438f6d905d30d551c51d758bdfeba498a4f04becc: Status 404 returned error can't find the container with id 1c65087c73021335c08e73e438f6d905d30d551c51d758bdfeba498a4f04becc Jul 21 21:01:41.115243 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:41.115218 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cghfw_7f05adfe-0e27-4ea2-8e61-4e03c16fba01/dns-node-resolver/0.log" Jul 21 21:01:41.214167 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:41.214134 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-4ptst" event={"ID":"cd4910b2-173b-425f-a555-0af5492f0dab","Type":"ContainerStarted","Data":"1c65087c73021335c08e73e438f6d905d30d551c51d758bdfeba498a4f04becc"} Jul 21 21:01:41.408001 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:41.407913 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-2l29r\" (UID: \"9148cdbf-145d-4850-8941-686d1a7db2cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:41.408406 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:41.408044 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 21:01:41.408406 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:41.408110 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls podName:9148cdbf-145d-4850-8941-686d1a7db2cd nodeName:}" failed. No retries permitted until 2026-07-21 21:01:49.408092518 +0000 UTC m=+89.018845897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-2l29r" (UID: "9148cdbf-145d-4850-8941-686d1a7db2cd") : secret "samples-operator-tls" not found Jul 21 21:01:42.113407 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:42.113379 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pmwz_65f51fb4-b664-45da-81ab-4a0d20506ad2/node-ca/0.log" Jul 21 21:01:42.215061 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:42.215029 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:42.215061 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:42.215071 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:42.215288 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:42.215160 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 21:01:42.215288 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:42.215193 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 21:01:42.215288 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:42.215220 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm podName:10107a18-038c-41a9-847d-c96604afd79f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:46.215201915 +0000 UTC m=+85.825955298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n2tht" (UID: "10107a18-038c-41a9-847d-c96604afd79f") : configmap "kube-rbac-proxy" not found Jul 21 21:01:42.215288 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:42.215255 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls podName:10107a18-038c-41a9-847d-c96604afd79f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:46.215236134 +0000 UTC m=+85.825989509 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n2tht" (UID: "10107a18-038c-41a9-847d-c96604afd79f") : secret "insights-runtime-extractor-tls" not found Jul 21 21:01:43.113160 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:43.113136 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7xpvd_b7561c51-58dc-4d33-89d8-974e4cc6601a/serve-healthcheck-canary/0.log" Jul 21 21:01:43.220997 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:43.220967 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-4ptst" event={"ID":"cd4910b2-173b-425f-a555-0af5492f0dab","Type":"ContainerStarted","Data":"2af4f2eb259213722b3da3a6dc20d8ef25729ffffff273327d8ea4f56b2bd2f6"} Jul 21 21:01:43.242175 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:43.242130 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-4ptst" podStartSLOduration=1.426082423 podStartE2EDuration="3.242116746s" totalImagesPullingTime="1.816034323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:01:43.241536927 +0000 UTC m=+82.852290311" watchObservedRunningTime="2026-07-21 21:01:43.242116746 +0000 UTC m=+82.852870129" Jul 21 21:01:44.022322 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:44.022284 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:44.022750 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:44.022734 2568 scope.go:117] "RemoveContainer" containerID="02c1bc4878b54979a7857678c033399e0170dc916f430d6ba95c4f495a5623e8" Jul 21 21:01:44.022933 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:44.022911 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ls9lw_openshift-console-operator(604ec39e-5292-4177-a045-5f639fc3d042)\"" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" podUID="604ec39e-5292-4177-a045-5f639fc3d042" Jul 21 21:01:46.190701 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:46.190664 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:01:46.191082 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:46.191026 2568 scope.go:117] "RemoveContainer" containerID="02c1bc4878b54979a7857678c033399e0170dc916f430d6ba95c4f495a5623e8" Jul 21 21:01:46.191172 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:46.191154 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ls9lw_openshift-console-operator(604ec39e-5292-4177-a045-5f639fc3d042)\"" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" podUID="604ec39e-5292-4177-a045-5f639fc3d042" Jul 21 21:01:46.247076 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:46.247043 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:46.247247 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:46.247084 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:46.247247 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:46.247177 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 21:01:46.247247 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:46.247193 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 21:01:46.247247 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:46.247248 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls podName:10107a18-038c-41a9-847d-c96604afd79f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:54.247230288 +0000 UTC m=+93.857983685 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n2tht" (UID: "10107a18-038c-41a9-847d-c96604afd79f") : secret "insights-runtime-extractor-tls" not found Jul 21 21:01:46.247496 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:01:46.247264 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm podName:10107a18-038c-41a9-847d-c96604afd79f nodeName:}" failed. No retries permitted until 2026-07-21 21:01:54.247257786 +0000 UTC m=+93.858011148 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n2tht" (UID: "10107a18-038c-41a9-847d-c96604afd79f") : configmap "kube-rbac-proxy" not found Jul 21 21:01:49.469028 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:49.468996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-2l29r\" (UID: \"9148cdbf-145d-4850-8941-686d1a7db2cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:49.471648 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:49.471615 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9148cdbf-145d-4850-8941-686d1a7db2cd-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-2l29r\" (UID: \"9148cdbf-145d-4850-8941-686d1a7db2cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:49.628692 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:49.628656 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" Jul 21 21:01:49.753431 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:49.753269 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r"] Jul 21 21:01:50.242883 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:50.242849 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" event={"ID":"9148cdbf-145d-4850-8941-686d1a7db2cd","Type":"ContainerStarted","Data":"c3db789796582426d3696a976422d3c6d924f9c9318578a8b4c41a7e7c3b5148"} Jul 21 21:01:52.250327 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:52.250285 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" event={"ID":"9148cdbf-145d-4850-8941-686d1a7db2cd","Type":"ContainerStarted","Data":"651278bd50d175305d7d583a75c90507a2de4394522a6cb40f8b398c787e253d"} Jul 21 21:01:52.250727 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:52.250358 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" event={"ID":"9148cdbf-145d-4850-8941-686d1a7db2cd","Type":"ContainerStarted","Data":"c01cbd40033488637b469f8d759b8765cf11a4d5ebbd6abceecbdffc03829e8c"} Jul 21 21:01:52.274523 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:52.274457 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-2l29r" podStartSLOduration=17.584755098 podStartE2EDuration="19.274440828s" totalImagesPullingTime="1.68968573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:01:52.273943406 +0000 UTC m=+91.884696790" watchObservedRunningTime="2026-07-21 21:01:52.274440828 +0000 UTC m=+91.885194212" Jul 21 21:01:54.312061 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:54.312021 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:54.312061 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:54.312063 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:54.312549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:54.312528 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10107a18-038c-41a9-847d-c96604afd79f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:54.314672 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:54.314645 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10107a18-038c-41a9-847d-c96604afd79f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2tht\" (UID: \"10107a18-038c-41a9-847d-c96604afd79f\") " pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:54.453021 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:54.452980 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n2tht" Jul 21 21:01:54.584581 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:54.584493 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n2tht"] Jul 21 21:01:54.587896 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:01:54.587868 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10107a18_038c_41a9_847d_c96604afd79f.slice/crio-57569ff0f2ba2638174b844b6168c602d5501a6a1220f7137a088794cfe3b3c9 WatchSource:0}: Error finding container 57569ff0f2ba2638174b844b6168c602d5501a6a1220f7137a088794cfe3b3c9: Status 404 returned error can't find the container with id 57569ff0f2ba2638174b844b6168c602d5501a6a1220f7137a088794cfe3b3c9 Jul 21 21:01:55.263256 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:55.263213 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2tht" event={"ID":"10107a18-038c-41a9-847d-c96604afd79f","Type":"ContainerStarted","Data":"57569ff0f2ba2638174b844b6168c602d5501a6a1220f7137a088794cfe3b3c9"} Jul 21 21:01:58.906518 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:01:58.906480 2568 scope.go:117] "RemoveContainer" containerID="02c1bc4878b54979a7857678c033399e0170dc916f430d6ba95c4f495a5623e8" Jul 21 21:02:00.285915 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:00.285843 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:02:00.286405 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:00.285982 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" event={"ID":"604ec39e-5292-4177-a045-5f639fc3d042","Type":"ContainerStarted","Data":"bc0f9bca0f21d707fe5686e94f1f6b065281e406fb715bd1f08a28b53008bb93"} Jul 21 21:02:00.286405 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:00.286391 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:02:00.287856 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:00.287826 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2tht" event={"ID":"10107a18-038c-41a9-847d-c96604afd79f","Type":"ContainerStarted","Data":"ee7958c62c06fb2d828fbf4debf3d7dc9383ce9abdfb7e44cd352717c131b73f"} Jul 21 21:02:00.293018 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:00.292995 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-ls9lw" Jul 21 21:02:01.169706 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:01.169663 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:02:01.169908 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:01.169758 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:02:01.172440 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:01.172407 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b-metrics-tls\") pod \"dns-default-qflql\" (UID: \"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b\") " pod="openshift-dns/dns-default-qflql" Jul 21 21:02:01.172764 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:01.172744 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") pod \"image-registry-8b4459dcb-z5rpn\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:02:01.448296 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:01.448216 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-58h5p\"" Jul 21 21:02:01.456403 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:01.456373 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:02:01.467151 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:01.467130 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wp8nm\"" Jul 21 21:02:01.475744 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:01.475717 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qflql" Jul 21 21:02:03.158269 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.158227 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf"] Jul 21 21:02:03.184823 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.182058 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.184823 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.183311 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf"] Jul 21 21:02:03.188219 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.187905 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jtn82" Jul 21 21:02:03.188219 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.187919 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 21 21:02:03.188421 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.188227 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 21 21:02:03.188480 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.188435 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 21 21:02:03.189262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.188773 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 21 21:02:03.189262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.188900 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 21 21:02:03.189262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.188987 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 21 21:02:03.189732 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.189709 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 21 21:02:03.274649 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.274617 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8b4459dcb-z5rpn"] Jul 21 21:02:03.277636 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:02:03.277606 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d3be26_c94d_47e2_90f0_93fa49ce1c8f.slice/crio-80c9d024a74401eba918de559c19c374bb654573102bf462f3288f7a5ce13628 WatchSource:0}: Error finding container 80c9d024a74401eba918de559c19c374bb654573102bf462f3288f7a5ce13628: Status 404 returned error can't find the container with id 80c9d024a74401eba918de559c19c374bb654573102bf462f3288f7a5ce13628 Jul 21 21:02:03.288209 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.288185 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-ca\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.288363 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.288344 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.288425 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.288380 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/413133bd-050f-428a-bc5c-633402dd57ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.288482 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.288426 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.288550 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.288502 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s8pr\" (UniqueName: \"kubernetes.io/projected/413133bd-050f-428a-bc5c-633402dd57ef-kube-api-access-2s8pr\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.288632 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.288547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-hub\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.290531 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.290512 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qflql"] Jul 21 21:02:03.306050 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.306018 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" event={"ID":"32d3be26-c94d-47e2-90f0-93fa49ce1c8f","Type":"ContainerStarted","Data":"80c9d024a74401eba918de559c19c374bb654573102bf462f3288f7a5ce13628"} Jul 21 21:02:03.389155 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.389066 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s8pr\" (UniqueName: \"kubernetes.io/projected/413133bd-050f-428a-bc5c-633402dd57ef-kube-api-access-2s8pr\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.389155 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.389101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-hub\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.389155 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.389137 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-ca\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.389428 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.389178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.389428 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.389198 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/413133bd-050f-428a-bc5c-633402dd57ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.389428 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.389215 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.390012 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.389952 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/413133bd-050f-428a-bc5c-633402dd57ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.391859 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.391835 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-ca\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.392320 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.392300 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.392412 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.392344 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-hub\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.392479 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.392460 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/413133bd-050f-428a-bc5c-633402dd57ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.401913 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.401893 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s8pr\" (UniqueName: \"kubernetes.io/projected/413133bd-050f-428a-bc5c-633402dd57ef-kube-api-access-2s8pr\") pod \"cluster-proxy-proxy-agent-6c66499876-6qrrf\" (UID: \"413133bd-050f-428a-bc5c-633402dd57ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.476068 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:02:03.476039 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad8bbb4c_8119_4c4d_8e7c_e4e5a5b86e4b.slice/crio-878e3ae8b2a1108ff12f5a0e03d97bd2976cb1d82e4c88dc7a1b2fe60714bb15 WatchSource:0}: Error finding container 878e3ae8b2a1108ff12f5a0e03d97bd2976cb1d82e4c88dc7a1b2fe60714bb15: Status 404 returned error can't find the container with id 878e3ae8b2a1108ff12f5a0e03d97bd2976cb1d82e4c88dc7a1b2fe60714bb15 Jul 21 21:02:03.506368 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.506339 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" Jul 21 21:02:03.663854 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:03.663680 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf"] Jul 21 21:02:03.666631 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:02:03.666581 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod413133bd_050f_428a_bc5c_633402dd57ef.slice/crio-21785c2be6d55c86de9a59dc031e2cfbd9823fe395b4166fb1f613a56ed07909 WatchSource:0}: Error finding container 21785c2be6d55c86de9a59dc031e2cfbd9823fe395b4166fb1f613a56ed07909: Status 404 returned error can't find the container with id 21785c2be6d55c86de9a59dc031e2cfbd9823fe395b4166fb1f613a56ed07909 Jul 21 21:02:04.316289 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:04.316244 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2tht" event={"ID":"10107a18-038c-41a9-847d-c96604afd79f","Type":"ContainerStarted","Data":"3a54473207ca51cc1263365dabe4e8dcaae8cee499845f9ed695499ce475d591"} Jul 21 21:02:04.317699 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:04.317666 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" event={"ID":"413133bd-050f-428a-bc5c-633402dd57ef","Type":"ContainerStarted","Data":"21785c2be6d55c86de9a59dc031e2cfbd9823fe395b4166fb1f613a56ed07909"} Jul 21 21:02:04.319133 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:04.319074 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qflql" event={"ID":"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b","Type":"ContainerStarted","Data":"878e3ae8b2a1108ff12f5a0e03d97bd2976cb1d82e4c88dc7a1b2fe60714bb15"} Jul 21 21:02:04.321235 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:04.321208 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" event={"ID":"32d3be26-c94d-47e2-90f0-93fa49ce1c8f","Type":"ContainerStarted","Data":"59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44"} Jul 21 21:02:04.321464 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:04.321439 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:02:04.343697 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:04.342945 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" podStartSLOduration=103.342924825 podStartE2EDuration="1m43.342924825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:02:04.34193163 +0000 UTC m=+103.952685014" watchObservedRunningTime="2026-07-21 21:02:04.342924825 +0000 UTC m=+103.953678209" Jul 21 21:02:10.345801 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:10.345758 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" event={"ID":"413133bd-050f-428a-bc5c-633402dd57ef","Type":"ContainerStarted","Data":"81a6f9277779cde2aae96104cc84b3d504dc5e63ae44776cd44fed08655bcef2"} Jul 21 21:02:10.346322 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:10.345810 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" event={"ID":"413133bd-050f-428a-bc5c-633402dd57ef","Type":"ContainerStarted","Data":"34638f9a187a8c9a1cbb7416f5813d8004c95ad8bcd6a11e990ac5e8a5c9c3a8"} Jul 21 21:02:10.346322 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:10.345827 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" event={"ID":"413133bd-050f-428a-bc5c-633402dd57ef","Type":"ContainerStarted","Data":"83dd0a17f6c11b40eea2d5459c0f34aea045d127d482f2d59a54b1f5b77f6b2d"} Jul 21 21:02:10.347677 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:10.347650 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qflql" event={"ID":"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b","Type":"ContainerStarted","Data":"e5d60d2e6d90e13e39eaabf3bdc99007faed69a11b6bc212e0869c92a879ec39"} Jul 21 21:02:10.347812 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:10.347744 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qflql" event={"ID":"ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b","Type":"ContainerStarted","Data":"b68bf59d6a6e8a07e5b83bd13ed1b0759e341012d0d2f279de3afa189c306eff"} Jul 21 21:02:10.347812 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:10.347772 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qflql" Jul 21 21:02:10.349614 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:10.349573 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2tht" event={"ID":"10107a18-038c-41a9-847d-c96604afd79f","Type":"ContainerStarted","Data":"430e067e88206633994af5f3698d0da7cb4a4cb521b9bd0240e73322cd5b5a9e"} Jul 21 21:02:10.376646 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:10.376580 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c66499876-6qrrf" podStartSLOduration=1.2699755160000001 podStartE2EDuration="7.376566086s" totalImagesPullingTime="6.10659057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:02:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:02:10.375941766 +0000 UTC m=+109.986695150" watchObservedRunningTime="2026-07-21 21:02:10.376566086 +0000 UTC m=+109.987319470" Jul 21 21:02:10.400650 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:10.400580 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qflql" podStartSLOduration=67.141679402 podStartE2EDuration="1m13.400563876s" totalImagesPullingTime="6.258884474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:02:10.399989259 +0000 UTC m=+110.010742643" watchObservedRunningTime="2026-07-21 21:02:10.400563876 +0000 UTC m=+110.011317260" Jul 21 21:02:10.423961 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:10.423910 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n2tht" podStartSLOduration=17.439157875 podStartE2EDuration="32.42389396s" totalImagesPullingTime="14.984736085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:01:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:02:10.422822151 +0000 UTC m=+110.033575533" watchObservedRunningTime="2026-07-21 21:02:10.42389396 +0000 UTC m=+110.034647343" Jul 21 21:02:20.355096 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:20.355065 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qflql" Jul 21 21:02:25.035905 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:25.035873 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8b4459dcb-z5rpn"] Jul 21 21:02:25.041628 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:25.041571 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:02:27.020057 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.020015 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-v6w5c"] Jul 21 21:02:27.026774 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.026739 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.030137 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.030104 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 21:02:27.030293 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.030169 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 21:02:27.030293 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.030277 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 21:02:27.031431 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.031409 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 21:02:27.031542 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.031452 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6hrnm\"" Jul 21 21:02:27.031542 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.031521 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 21:02:27.031737 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.031723 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 21:02:27.071138 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.071103 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnvvs\" (UniqueName: \"kubernetes.io/projected/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-kube-api-access-wnvvs\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.071342 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.071179 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-root\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.071342 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.071276 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-accelerators-collector-config\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.071342 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.071321 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-textfile\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.071496 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.071373 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-tls\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.071496 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.071418 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-sys\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.071496 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.071449 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-wtmp\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.071496 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.071470 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.071496 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.071493 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-metrics-client-ca\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.172819 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.172787 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-sys\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.172819 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.172825 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-wtmp\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173064 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.172851 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173064 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.172881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-metrics-client-ca\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173064 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.172902 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-sys\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173064 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.172921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnvvs\" (UniqueName: \"kubernetes.io/projected/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-kube-api-access-wnvvs\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173064 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.172967 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-root\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173064 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.173015 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-accelerators-collector-config\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173064 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.173020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-wtmp\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173064 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.173042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-textfile\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173064 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.173054 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-root\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173433 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.173177 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-tls\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173433 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.173405 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-textfile\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173564 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.173544 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-accelerators-collector-config\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.173564 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.173554 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-metrics-client-ca\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.175420 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.175393 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.175525 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.175502 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-node-exporter-tls\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.182394 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.182373 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnvvs\" (UniqueName: \"kubernetes.io/projected/e0e5cd46-da6d-4eb3-9ff7-dd3b62961850-kube-api-access-wnvvs\") pod \"node-exporter-v6w5c\" (UID: \"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850\") " pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.337123 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.337083 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v6w5c" Jul 21 21:02:27.350507 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:02:27.350474 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e5cd46_da6d_4eb3_9ff7_dd3b62961850.slice/crio-e4c2e8f50541140f95400d08a2a593f7a1b760e50aed4624d45b593b4ba8a99d WatchSource:0}: Error finding container e4c2e8f50541140f95400d08a2a593f7a1b760e50aed4624d45b593b4ba8a99d: Status 404 returned error can't find the container with id e4c2e8f50541140f95400d08a2a593f7a1b760e50aed4624d45b593b4ba8a99d Jul 21 21:02:27.406155 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:27.406119 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v6w5c" event={"ID":"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850","Type":"ContainerStarted","Data":"e4c2e8f50541140f95400d08a2a593f7a1b760e50aed4624d45b593b4ba8a99d"} Jul 21 21:02:28.040107 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.040073 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 21:02:28.044536 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.044505 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.047352 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.047324 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 21 21:02:28.048542 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.048519 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 21 21:02:28.049637 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.049290 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4lz4f\"" Jul 21 21:02:28.049637 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.049387 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 21 21:02:28.049637 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.049418 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 21 21:02:28.049637 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.049465 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 21 21:02:28.050627 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.050067 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 21 21:02:28.050627 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.050551 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 21 21:02:28.050627 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.050385 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 21 21:02:28.052012 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.050939 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 21 21:02:28.062698 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.062673 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 21:02:28.079803 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.079769 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/07189de5-d912-41d5-9b48-bdbd98d9e31a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.079981 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.079815 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07189de5-d912-41d5-9b48-bdbd98d9e31a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.079981 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.079845 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-web-config\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.079981 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.079869 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.079981 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.079914 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/07189de5-d912-41d5-9b48-bdbd98d9e31a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.080185 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.079948 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.080185 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.080019 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/07189de5-d912-41d5-9b48-bdbd98d9e31a-config-out\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.080185 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.080091 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf2wg\" (UniqueName: \"kubernetes.io/projected/07189de5-d912-41d5-9b48-bdbd98d9e31a-kube-api-access-mf2wg\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.080185 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.080139 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07189de5-d912-41d5-9b48-bdbd98d9e31a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.080378 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.080197 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.080378 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.080254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-config-volume\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.080378 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.080281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.080378 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.080307 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.181610 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.181563 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07189de5-d912-41d5-9b48-bdbd98d9e31a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.181790 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.181629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.181790 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.181671 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-config-volume\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.181790 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.181699 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.181790 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:02:28.181749 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07189de5-d912-41d5-9b48-bdbd98d9e31a-alertmanager-trusted-ca-bundle podName:07189de5-d912-41d5-9b48-bdbd98d9e31a nodeName:}" failed. No retries permitted until 2026-07-21 21:02:28.681719519 +0000 UTC m=+128.292472896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/07189de5-d912-41d5-9b48-bdbd98d9e31a-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "07189de5-d912-41d5-9b48-bdbd98d9e31a") : configmap references non-existent config key: ca-bundle.crt Jul 21 21:02:28.182087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.181798 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.182087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.181848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/07189de5-d912-41d5-9b48-bdbd98d9e31a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.182087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.181880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07189de5-d912-41d5-9b48-bdbd98d9e31a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.182087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.181912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-web-config\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.182087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.181936 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.182087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.181981 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/07189de5-d912-41d5-9b48-bdbd98d9e31a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.182087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.182018 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.182087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.182052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/07189de5-d912-41d5-9b48-bdbd98d9e31a-config-out\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.182087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.182081 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf2wg\" (UniqueName: \"kubernetes.io/projected/07189de5-d912-41d5-9b48-bdbd98d9e31a-kube-api-access-mf2wg\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.182559 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:02:28.181803 2568 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 21 21:02:28.182559 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:02:28.182222 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-main-tls podName:07189de5-d912-41d5-9b48-bdbd98d9e31a nodeName:}" failed. No retries permitted until 2026-07-21 21:02:28.682196143 +0000 UTC m=+128.292949512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "07189de5-d912-41d5-9b48-bdbd98d9e31a") : secret "alertmanager-main-tls" not found Jul 21 21:02:28.182884 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.182821 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07189de5-d912-41d5-9b48-bdbd98d9e31a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.183371 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.183315 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/07189de5-d912-41d5-9b48-bdbd98d9e31a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.185308 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.185281 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-config-volume\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.185959 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.185710 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.185959 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.185923 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/07189de5-d912-41d5-9b48-bdbd98d9e31a-config-out\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.185959 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.185932 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.186638 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.186614 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/07189de5-d912-41d5-9b48-bdbd98d9e31a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.187513 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.187483 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.188669 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.188624 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-web-config\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.188872 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.188830 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.191178 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.191153 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf2wg\" (UniqueName: \"kubernetes.io/projected/07189de5-d912-41d5-9b48-bdbd98d9e31a-kube-api-access-mf2wg\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.686783 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.686678 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07189de5-d912-41d5-9b48-bdbd98d9e31a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.686783 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.686738 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.687525 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.687494 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07189de5-d912-41d5-9b48-bdbd98d9e31a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.689322 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.689304 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/07189de5-d912-41d5-9b48-bdbd98d9e31a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"07189de5-d912-41d5-9b48-bdbd98d9e31a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:28.960265 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:28.960177 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 21:02:29.100270 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:29.100247 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 21:02:29.102829 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:02:29.102790 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07189de5_d912_41d5_9b48_bdbd98d9e31a.slice/crio-5f374aa9eb753b3e2a345764265a6a2eeba6061cdae4ac2a4c9a02c41927d0b4 WatchSource:0}: Error finding container 5f374aa9eb753b3e2a345764265a6a2eeba6061cdae4ac2a4c9a02c41927d0b4: Status 404 returned error can't find the container with id 5f374aa9eb753b3e2a345764265a6a2eeba6061cdae4ac2a4c9a02c41927d0b4 Jul 21 21:02:29.414558 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:29.414525 2568 generic.go:356] "Generic (PLEG): container finished" podID="e0e5cd46-da6d-4eb3-9ff7-dd3b62961850" containerID="0c6dc5c07461eaf1e85cf16bd5cae6b3e6b5b03ed78277052d324047f02e03fe" exitCode=0 Jul 21 21:02:29.414739 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:29.414619 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v6w5c" event={"ID":"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850","Type":"ContainerDied","Data":"0c6dc5c07461eaf1e85cf16bd5cae6b3e6b5b03ed78277052d324047f02e03fe"} Jul 21 21:02:29.416014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:29.415987 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"07189de5-d912-41d5-9b48-bdbd98d9e31a","Type":"ContainerStarted","Data":"5f374aa9eb753b3e2a345764265a6a2eeba6061cdae4ac2a4c9a02c41927d0b4"} Jul 21 21:02:30.422118 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:30.422078 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v6w5c" event={"ID":"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850","Type":"ContainerStarted","Data":"be49824a6bd0b028d2d0d2fd326d6c25b9c5dabff052ab51b3e4df23b536f0fe"} Jul 21 21:02:30.422577 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:30.422147 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v6w5c" event={"ID":"e0e5cd46-da6d-4eb3-9ff7-dd3b62961850","Type":"ContainerStarted","Data":"bbf7219965246c626b19b43fac2ceee9b84fe43f4b70b0a966935041704fd1fb"} Jul 21 21:02:30.447653 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:30.447571 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-v6w5c" podStartSLOduration=3.298585638 podStartE2EDuration="4.447554188s" totalImagesPullingTime="1.078507856s" totalInitContainerRuntime="70.460694ms" isStatefulPod=true podCreationTimestamp="2026-07-21 21:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:02:30.445973762 +0000 UTC m=+130.056727147" watchObservedRunningTime="2026-07-21 21:02:30.447554188 +0000 UTC m=+130.058307571" Jul 21 21:02:30.803893 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:30.803858 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:02:30.806377 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:30.806341 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/819c1153-e1db-4114-8e57-eb79d24d8db9-metrics-certs\") pod \"network-metrics-daemon-ckmm7\" (UID: \"819c1153-e1db-4114-8e57-eb79d24d8db9\") " pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:02:31.037874 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.037847 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cvwws\"" Jul 21 21:02:31.046354 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.046330 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckmm7" Jul 21 21:02:31.196271 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.196114 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ckmm7"] Jul 21 21:02:31.198689 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:02:31.198663 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod819c1153_e1db_4114_8e57_eb79d24d8db9.slice/crio-c04540a45de80ec889b30edd7e1c6e494adaff220109990ad20d82d99cf536d8 WatchSource:0}: Error finding container c04540a45de80ec889b30edd7e1c6e494adaff220109990ad20d82d99cf536d8: Status 404 returned error can't find the container with id c04540a45de80ec889b30edd7e1c6e494adaff220109990ad20d82d99cf536d8 Jul 21 21:02:31.427046 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.426943 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ckmm7" event={"ID":"819c1153-e1db-4114-8e57-eb79d24d8db9","Type":"ContainerStarted","Data":"c04540a45de80ec889b30edd7e1c6e494adaff220109990ad20d82d99cf536d8"} Jul 21 21:02:31.428486 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.428456 2568 generic.go:356] "Generic (PLEG): container finished" podID="07189de5-d912-41d5-9b48-bdbd98d9e31a" containerID="2bf625da4021b805fd8961507acfc536b9750ddc204442860a36a29ff6f7754d" exitCode=0 Jul 21 21:02:31.428659 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.428538 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"07189de5-d912-41d5-9b48-bdbd98d9e31a","Type":"ContainerDied","Data":"2bf625da4021b805fd8961507acfc536b9750ddc204442860a36a29ff6f7754d"} Jul 21 21:02:31.748676 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.748599 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7"] Jul 21 21:02:31.762769 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.762736 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7"] Jul 21 21:02:31.762945 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.762870 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" Jul 21 21:02:31.765632 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.765606 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 21 21:02:31.765775 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.765696 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hz4zs\"" Jul 21 21:02:31.813084 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.813058 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-57dj7\" (UID: \"f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" Jul 21 21:02:31.813234 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.813207 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kttt\" (UniqueName: \"kubernetes.io/projected/f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e-kube-api-access-6kttt\") pod \"monitoring-plugin-7b7b7bc8bf-57dj7\" (UID: \"f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" Jul 21 21:02:31.914107 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.914064 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kttt\" (UniqueName: \"kubernetes.io/projected/f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e-kube-api-access-6kttt\") pod \"monitoring-plugin-7b7b7bc8bf-57dj7\" (UID: \"f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" Jul 21 21:02:31.914107 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.914110 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-57dj7\" (UID: \"f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" Jul 21 21:02:31.916797 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.916769 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-57dj7\" (UID: \"f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" Jul 21 21:02:31.923661 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:31.923634 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kttt\" (UniqueName: \"kubernetes.io/projected/f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e-kube-api-access-6kttt\") pod \"monitoring-plugin-7b7b7bc8bf-57dj7\" (UID: \"f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" Jul 21 21:02:32.072923 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:32.072880 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" Jul 21 21:02:32.245948 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:32.245912 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7"] Jul 21 21:02:32.249031 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:02:32.248999 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf29b55a7_ff87_49d7_93d6_b1f0fc8ac44e.slice/crio-fb77909433c7f6633a7e0d4c803eb8cb10c79560d22ccd101ae855cba6f55a49 WatchSource:0}: Error finding container fb77909433c7f6633a7e0d4c803eb8cb10c79560d22ccd101ae855cba6f55a49: Status 404 returned error can't find the container with id fb77909433c7f6633a7e0d4c803eb8cb10c79560d22ccd101ae855cba6f55a49 Jul 21 21:02:32.434665 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:32.434515 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" event={"ID":"f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e","Type":"ContainerStarted","Data":"fb77909433c7f6633a7e0d4c803eb8cb10c79560d22ccd101ae855cba6f55a49"} Jul 21 21:02:33.441284 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:33.441249 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ckmm7" event={"ID":"819c1153-e1db-4114-8e57-eb79d24d8db9","Type":"ContainerStarted","Data":"9c18b865c0ef140e52db37ed482527bb756e7b2a8a340f34463712a794499bc3"} Jul 21 21:02:34.447571 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:34.447467 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"07189de5-d912-41d5-9b48-bdbd98d9e31a","Type":"ContainerStarted","Data":"828c22361e2e94c45ce5fdbc3169577ae18d0d2d0c42371f5a403eb810cee563"} Jul 21 21:02:34.448005 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:34.447618 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"07189de5-d912-41d5-9b48-bdbd98d9e31a","Type":"ContainerStarted","Data":"9dadbaa903102cc970747f54bd45f4664ad2ee079549c513e24faedcb4472ba8"} Jul 21 21:02:34.448005 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:34.447668 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"07189de5-d912-41d5-9b48-bdbd98d9e31a","Type":"ContainerStarted","Data":"c9c3ba5eba32cf75aa836ad8f43bab3929d071cfd18af87e5991608064cac336"} Jul 21 21:02:34.448005 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:34.447713 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"07189de5-d912-41d5-9b48-bdbd98d9e31a","Type":"ContainerStarted","Data":"d3b9f67c5f34cec71dc1cee80a4bdccdecb7e6d037af705e251575e49388e20f"} Jul 21 21:02:34.448005 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:34.447755 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"07189de5-d912-41d5-9b48-bdbd98d9e31a","Type":"ContainerStarted","Data":"e237a3e7acb3e269f2fa220cb1dafc56c6236790b6a44338711e13428a097eb0"} Jul 21 21:02:34.448954 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:34.448929 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" event={"ID":"f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e","Type":"ContainerStarted","Data":"3c893578cf7b26b880285b99c13232aaf6a0d4460cf469a4c5ca8f3f61407e39"} Jul 21 21:02:34.449129 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:34.449101 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" Jul 21 21:02:34.450657 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:34.450627 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ckmm7" event={"ID":"819c1153-e1db-4114-8e57-eb79d24d8db9","Type":"ContainerStarted","Data":"a70064ceb97cd070b686e064fc9a0ed0c04a6faaa963aad41ff54c07299d19a4"} Jul 21 21:02:34.454328 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:34.454263 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" Jul 21 21:02:34.468500 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:34.468453 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-57dj7" podStartSLOduration=1.771433456 podStartE2EDuration="3.468437797s" totalImagesPullingTime="1.697004341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:02:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:02:34.467217067 +0000 UTC m=+134.077970452" watchObservedRunningTime="2026-07-21 21:02:34.468437797 +0000 UTC m=+134.079191182" Jul 21 21:02:34.483360 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:34.483312 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ckmm7" podStartSLOduration=131.376544029 podStartE2EDuration="2m13.483291342s" totalImagesPullingTime="2.106747313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:02:34.48302471 +0000 UTC m=+134.093778095" watchObservedRunningTime="2026-07-21 21:02:34.483291342 +0000 UTC m=+134.094044727" Jul 21 21:02:35.457780 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:35.457680 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"07189de5-d912-41d5-9b48-bdbd98d9e31a","Type":"ContainerStarted","Data":"ef65b1fff7d633e564ec25ed8840b7d0de5f2d107f434c88f48c2261474db189"} Jul 21 21:02:35.492617 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:35.492452 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.264594734 podStartE2EDuration="7.492431854s" totalImagesPullingTime="5.214472534s" totalInitContainerRuntime="13.364586ms" isStatefulPod=false podCreationTimestamp="2026-07-21 21:02:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:02:35.49099827 +0000 UTC m=+135.101751653" watchObservedRunningTime="2026-07-21 21:02:35.492431854 +0000 UTC m=+135.103185239" Jul 21 21:02:50.056772 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.056725 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" podUID="32d3be26-c94d-47e2-90f0-93fa49ce1c8f" containerName="registry" containerID="cri-o://59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44" gracePeriod=30 Jul 21 21:02:50.299535 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.299507 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:02:50.364687 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.364579 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-ca-trust-extracted\") pod \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " Jul 21 21:02:50.364687 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.364640 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-installation-pull-secrets\") pod \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " Jul 21 21:02:50.364687 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.364675 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-image-registry-private-configuration\") pod \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " Jul 21 21:02:50.365000 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.364711 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz2x4\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-kube-api-access-vz2x4\") pod \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " Jul 21 21:02:50.365000 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.364727 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") pod \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " Jul 21 21:02:50.365000 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.364765 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-bound-sa-token\") pod \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " Jul 21 21:02:50.365000 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.364907 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-trusted-ca\") pod \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " Jul 21 21:02:50.365000 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.364975 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-certificates\") pod \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\" (UID: \"32d3be26-c94d-47e2-90f0-93fa49ce1c8f\") " Jul 21 21:02:50.366113 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.366076 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "32d3be26-c94d-47e2-90f0-93fa49ce1c8f" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 21:02:50.366345 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.366304 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "32d3be26-c94d-47e2-90f0-93fa49ce1c8f" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 21:02:50.367460 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.367419 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "32d3be26-c94d-47e2-90f0-93fa49ce1c8f" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:02:50.367555 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.367505 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "32d3be26-c94d-47e2-90f0-93fa49ce1c8f" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:02:50.367785 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.367761 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "32d3be26-c94d-47e2-90f0-93fa49ce1c8f" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:02:50.367851 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.367822 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "32d3be26-c94d-47e2-90f0-93fa49ce1c8f" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:02:50.367889 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.367841 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-kube-api-access-vz2x4" (OuterVolumeSpecName: "kube-api-access-vz2x4") pod "32d3be26-c94d-47e2-90f0-93fa49ce1c8f" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f"). InnerVolumeSpecName "kube-api-access-vz2x4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:02:50.373470 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.373443 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "32d3be26-c94d-47e2-90f0-93fa49ce1c8f" (UID: "32d3be26-c94d-47e2-90f0-93fa49ce1c8f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:02:50.465830 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.465778 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz2x4\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-kube-api-access-vz2x4\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:02:50.465830 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.465824 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:02:50.465830 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.465840 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-bound-sa-token\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:02:50.466057 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.465852 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-trusted-ca\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:02:50.466057 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.465865 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-registry-certificates\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:02:50.466057 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.465876 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-ca-trust-extracted\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:02:50.466057 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.465887 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-installation-pull-secrets\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:02:50.466057 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.465900 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/32d3be26-c94d-47e2-90f0-93fa49ce1c8f-image-registry-private-configuration\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:02:50.518175 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.518140 2568 generic.go:356] "Generic (PLEG): container finished" podID="32d3be26-c94d-47e2-90f0-93fa49ce1c8f" containerID="59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44" exitCode=0 Jul 21 21:02:50.518313 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.518190 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" Jul 21 21:02:50.518313 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.518233 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" event={"ID":"32d3be26-c94d-47e2-90f0-93fa49ce1c8f","Type":"ContainerDied","Data":"59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44"} Jul 21 21:02:50.518313 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.518310 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8b4459dcb-z5rpn" event={"ID":"32d3be26-c94d-47e2-90f0-93fa49ce1c8f","Type":"ContainerDied","Data":"80c9d024a74401eba918de559c19c374bb654573102bf462f3288f7a5ce13628"} Jul 21 21:02:50.518429 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.518332 2568 scope.go:117] "RemoveContainer" containerID="59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44" Jul 21 21:02:50.526996 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.526979 2568 scope.go:117] "RemoveContainer" containerID="59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44" Jul 21 21:02:50.527286 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:02:50.527265 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44\": container with ID starting with 59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44 not found: ID does not exist" containerID="59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44" Jul 21 21:02:50.527361 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.527308 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44"} err="failed to get container status \"59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44\": rpc error: code = NotFound desc = could not find container \"59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44\": container with ID starting with 59d2fc4701b529cccc391e5eaf2011faee0b12cb8eea4eba40d7720058283d44 not found: ID does not exist" Jul 21 21:02:50.539483 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.539453 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8b4459dcb-z5rpn"] Jul 21 21:02:50.543310 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.543286 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8b4459dcb-z5rpn"] Jul 21 21:02:50.910471 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:02:50.910441 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32d3be26-c94d-47e2-90f0-93fa49ce1c8f" path="/var/lib/kubelet/pods/32d3be26-c94d-47e2-90f0-93fa49ce1c8f/volumes" Jul 21 21:03:51.281769 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.281732 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-65c85bcf49-csjs4"] Jul 21 21:03:51.282176 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.281990 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32d3be26-c94d-47e2-90f0-93fa49ce1c8f" containerName="registry" Jul 21 21:03:51.282176 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.282000 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="32d3be26-c94d-47e2-90f0-93fa49ce1c8f" containerName="registry" Jul 21 21:03:51.282176 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.282045 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="32d3be26-c94d-47e2-90f0-93fa49ce1c8f" containerName="registry" Jul 21 21:03:51.285017 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.284999 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.287576 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.287556 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 21 21:03:51.287713 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.287614 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 21 21:03:51.287713 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.287611 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 21 21:03:51.287839 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.287815 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 21 21:03:51.287936 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.287821 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 21 21:03:51.288149 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.288128 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-ns57j\"" Jul 21 21:03:51.293195 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.293173 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 21 21:03:51.298046 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.298026 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65c85bcf49-csjs4"] Jul 21 21:03:51.301688 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.301667 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c626b62c-02d3-4e14-a4a9-d54540b5ef54-serving-certs-ca-bundle\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.301782 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.301693 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-secret-telemeter-client\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.301782 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.301718 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.301782 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.301735 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c626b62c-02d3-4e14-a4a9-d54540b5ef54-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.301896 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.301781 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh5wz\" (UniqueName: \"kubernetes.io/projected/c626b62c-02d3-4e14-a4a9-d54540b5ef54-kube-api-access-jh5wz\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.301896 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.301873 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-telemeter-client-tls\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.301977 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.301911 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-federate-client-tls\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.302036 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.302003 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c626b62c-02d3-4e14-a4a9-d54540b5ef54-metrics-client-ca\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.402602 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.402563 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-telemeter-client-tls\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.402779 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.402628 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-federate-client-tls\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.402779 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.402668 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c626b62c-02d3-4e14-a4a9-d54540b5ef54-metrics-client-ca\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.402779 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.402711 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c626b62c-02d3-4e14-a4a9-d54540b5ef54-serving-certs-ca-bundle\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.402779 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.402736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-secret-telemeter-client\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.402779 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.402778 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.403038 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.402806 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c626b62c-02d3-4e14-a4a9-d54540b5ef54-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.403038 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.402843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh5wz\" (UniqueName: \"kubernetes.io/projected/c626b62c-02d3-4e14-a4a9-d54540b5ef54-kube-api-access-jh5wz\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.403722 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.403689 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c626b62c-02d3-4e14-a4a9-d54540b5ef54-serving-certs-ca-bundle\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.403866 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.403835 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c626b62c-02d3-4e14-a4a9-d54540b5ef54-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.403866 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.403841 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c626b62c-02d3-4e14-a4a9-d54540b5ef54-metrics-client-ca\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.405375 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.405346 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-secret-telemeter-client\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.405375 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.405357 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-telemeter-client-tls\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.405806 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.405784 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.405851 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.405807 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c626b62c-02d3-4e14-a4a9-d54540b5ef54-federate-client-tls\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.410740 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.410717 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh5wz\" (UniqueName: \"kubernetes.io/projected/c626b62c-02d3-4e14-a4a9-d54540b5ef54-kube-api-access-jh5wz\") pod \"telemeter-client-65c85bcf49-csjs4\" (UID: \"c626b62c-02d3-4e14-a4a9-d54540b5ef54\") " pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.595065 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.595029 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" Jul 21 21:03:51.727419 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:51.727391 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65c85bcf49-csjs4"] Jul 21 21:03:51.733246 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:03:51.733218 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc626b62c_02d3_4e14_a4a9_d54540b5ef54.slice/crio-a88bbe1e9dcbd070311c6030738335c8ea4702c6b54911279bb281b092e7e04b WatchSource:0}: Error finding container a88bbe1e9dcbd070311c6030738335c8ea4702c6b54911279bb281b092e7e04b: Status 404 returned error can't find the container with id a88bbe1e9dcbd070311c6030738335c8ea4702c6b54911279bb281b092e7e04b Jul 21 21:03:52.739435 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:52.739397 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" event={"ID":"c626b62c-02d3-4e14-a4a9-d54540b5ef54","Type":"ContainerStarted","Data":"a88bbe1e9dcbd070311c6030738335c8ea4702c6b54911279bb281b092e7e04b"} Jul 21 21:03:53.745601 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:53.745558 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" event={"ID":"c626b62c-02d3-4e14-a4a9-d54540b5ef54","Type":"ContainerStarted","Data":"1c5f6b75c4dcc6f19529c27da38eddefd1eac241458249ce61c73450c6427b4e"} Jul 21 21:03:54.751240 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:54.751200 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" event={"ID":"c626b62c-02d3-4e14-a4a9-d54540b5ef54","Type":"ContainerStarted","Data":"238f615ef8a282915666d0d6d88b5047f08100c3031b828f3cddf1ce0fec8e91"} Jul 21 21:03:54.751657 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:54.751306 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" event={"ID":"c626b62c-02d3-4e14-a4a9-d54540b5ef54","Type":"ContainerStarted","Data":"f7b37e5c0c5b94a906c02e3fc4b36eb541a58f94bdb882760f61d0a6797d2a63"} Jul 21 21:03:54.776793 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:03:54.776741 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-65c85bcf49-csjs4" podStartSLOduration=1.885566033 podStartE2EDuration="3.776724454s" totalImagesPullingTime="1.891158421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:03:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:03:54.773818423 +0000 UTC m=+214.384571807" watchObservedRunningTime="2026-07-21 21:03:54.776724454 +0000 UTC m=+214.387477838" Jul 21 21:05:20.810548 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:20.810517 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:05:20.816168 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:20.816136 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:05:20.816451 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:20.816429 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:05:20.823700 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:20.823679 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 21:05:20.824332 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:20.824311 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:05:44.599665 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.599632 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr"] Jul 21 21:05:44.602987 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.602969 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr" Jul 21 21:05:44.606268 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.606239 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 21:05:44.606373 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.606304 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 21:05:44.606924 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.606910 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-bqzbb\"" Jul 21 21:05:44.614229 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.614206 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr"] Jul 21 21:05:44.715864 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.715821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b3561de-d959-4f14-9512-680348505e81-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s5xxr\" (UID: \"0b3561de-d959-4f14-9512-680348505e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr" Jul 21 21:05:44.715975 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.715897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpt27\" (UniqueName: \"kubernetes.io/projected/0b3561de-d959-4f14-9512-680348505e81-kube-api-access-dpt27\") pod \"cert-manager-cainjector-559cdd8c87-s5xxr\" (UID: \"0b3561de-d959-4f14-9512-680348505e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr" Jul 21 21:05:44.816372 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.816350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpt27\" (UniqueName: \"kubernetes.io/projected/0b3561de-d959-4f14-9512-680348505e81-kube-api-access-dpt27\") pod \"cert-manager-cainjector-559cdd8c87-s5xxr\" (UID: \"0b3561de-d959-4f14-9512-680348505e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr" Jul 21 21:05:44.816496 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.816478 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b3561de-d959-4f14-9512-680348505e81-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s5xxr\" (UID: \"0b3561de-d959-4f14-9512-680348505e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr" Jul 21 21:05:44.825944 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.825918 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpt27\" (UniqueName: \"kubernetes.io/projected/0b3561de-d959-4f14-9512-680348505e81-kube-api-access-dpt27\") pod \"cert-manager-cainjector-559cdd8c87-s5xxr\" (UID: \"0b3561de-d959-4f14-9512-680348505e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr" Jul 21 21:05:44.826031 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.825969 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b3561de-d959-4f14-9512-680348505e81-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s5xxr\" (UID: \"0b3561de-d959-4f14-9512-680348505e81\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr" Jul 21 21:05:44.911317 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:44.911248 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr" Jul 21 21:05:45.038719 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:45.038649 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr"] Jul 21 21:05:45.043192 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:05:45.043162 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b3561de_d959_4f14_9512_680348505e81.slice/crio-7d7f44c2930712211de28e5bcc9cd31fe9e2bb9c6fe4bceead8460e1495262dc WatchSource:0}: Error finding container 7d7f44c2930712211de28e5bcc9cd31fe9e2bb9c6fe4bceead8460e1495262dc: Status 404 returned error can't find the container with id 7d7f44c2930712211de28e5bcc9cd31fe9e2bb9c6fe4bceead8460e1495262dc Jul 21 21:05:45.045164 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:45.045150 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 21:05:45.136054 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:45.136017 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr" event={"ID":"0b3561de-d959-4f14-9512-680348505e81","Type":"ContainerStarted","Data":"7d7f44c2930712211de28e5bcc9cd31fe9e2bb9c6fe4bceead8460e1495262dc"} Jul 21 21:05:49.154907 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:49.154875 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr" event={"ID":"0b3561de-d959-4f14-9512-680348505e81","Type":"ContainerStarted","Data":"e2a2bd40db8245492aca181de4997c10879a548bceebcbb51627e74552879e67"} Jul 21 21:05:49.175067 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:05:49.175019 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s5xxr" podStartSLOduration=1.628290255 podStartE2EDuration="5.175004692s" totalImagesPullingTime="3.546714437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:05:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:05:49.173994779 +0000 UTC m=+328.784748177" watchObservedRunningTime="2026-07-21 21:05:49.175004692 +0000 UTC m=+328.785758076" Jul 21 21:06:14.924022 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:14.923990 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj"] Jul 21 21:06:14.927109 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:14.927093 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:14.929820 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:14.929796 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 21:06:14.929963 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:14.929833 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 21:06:14.929963 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:14.929840 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 21:06:14.929963 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:14.929868 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 21:06:14.930159 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:14.930146 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ddkqb\"" Jul 21 21:06:14.944769 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:14.944748 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj"] Jul 21 21:06:15.038335 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.038297 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb9a9560-e187-4512-9f0b-5cdb186f1fab-tmp\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.038485 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.038347 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fb9a9560-e187-4512-9f0b-5cdb186f1fab-manifests\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.038485 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.038375 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fb9a9560-e187-4512-9f0b-5cdb186f1fab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.038485 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.038417 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrdc7\" (UniqueName: \"kubernetes.io/projected/fb9a9560-e187-4512-9f0b-5cdb186f1fab-kube-api-access-lrdc7\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.038485 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.038444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9a9560-e187-4512-9f0b-5cdb186f1fab-webhook-cert\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.139214 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.139183 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb9a9560-e187-4512-9f0b-5cdb186f1fab-tmp\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.139386 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.139220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fb9a9560-e187-4512-9f0b-5cdb186f1fab-manifests\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.139386 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.139242 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fb9a9560-e187-4512-9f0b-5cdb186f1fab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.139386 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.139271 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrdc7\" (UniqueName: \"kubernetes.io/projected/fb9a9560-e187-4512-9f0b-5cdb186f1fab-kube-api-access-lrdc7\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.139386 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.139300 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9a9560-e187-4512-9f0b-5cdb186f1fab-webhook-cert\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.139662 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.139636 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fb9a9560-e187-4512-9f0b-5cdb186f1fab-manifests\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.141554 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.141526 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb9a9560-e187-4512-9f0b-5cdb186f1fab-tmp\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.141762 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.141743 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fb9a9560-e187-4512-9f0b-5cdb186f1fab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.141809 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.141785 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9a9560-e187-4512-9f0b-5cdb186f1fab-webhook-cert\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.147874 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.147853 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrdc7\" (UniqueName: \"kubernetes.io/projected/fb9a9560-e187-4512-9f0b-5cdb186f1fab-kube-api-access-lrdc7\") pod \"opendatahub-operator-controller-manager-694dcbb678-jq7lj\" (UID: \"fb9a9560-e187-4512-9f0b-5cdb186f1fab\") " pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.236825 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.236757 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:15.369077 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:15.369043 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj"] Jul 21 21:06:15.372925 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:06:15.372894 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9a9560_e187_4512_9f0b_5cdb186f1fab.slice/crio-de2d8a6f198aeb948cab2928ce37a02d7b89ea87aec927a966ea5a4d9a598580 WatchSource:0}: Error finding container de2d8a6f198aeb948cab2928ce37a02d7b89ea87aec927a966ea5a4d9a598580: Status 404 returned error can't find the container with id de2d8a6f198aeb948cab2928ce37a02d7b89ea87aec927a966ea5a4d9a598580 Jul 21 21:06:16.252078 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:16.252034 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" event={"ID":"fb9a9560-e187-4512-9f0b-5cdb186f1fab","Type":"ContainerStarted","Data":"de2d8a6f198aeb948cab2928ce37a02d7b89ea87aec927a966ea5a4d9a598580"} Jul 21 21:06:19.266826 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:19.266789 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" event={"ID":"fb9a9560-e187-4512-9f0b-5cdb186f1fab","Type":"ContainerStarted","Data":"cb0e2018ac40eee155f02da283e63b4b3647044bf79b2cf3dca91a037830b57f"} Jul 21 21:06:20.271905 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:20.271868 2568 generic.go:356] "Generic (PLEG): container finished" podID="fb9a9560-e187-4512-9f0b-5cdb186f1fab" containerID="cb0e2018ac40eee155f02da283e63b4b3647044bf79b2cf3dca91a037830b57f" exitCode=0 Jul 21 21:06:20.272279 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:20.271960 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" event={"ID":"fb9a9560-e187-4512-9f0b-5cdb186f1fab","Type":"ContainerDied","Data":"cb0e2018ac40eee155f02da283e63b4b3647044bf79b2cf3dca91a037830b57f"} Jul 21 21:06:21.278042 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:21.278009 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" event={"ID":"fb9a9560-e187-4512-9f0b-5cdb186f1fab","Type":"ContainerStarted","Data":"7a7b42d4ed7595392f519b42a3f9e57a3b1f4b86061b2678a0c8c5a9dc4335a6"} Jul 21 21:06:21.278405 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:21.278134 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:21.301029 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:21.300970 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" podStartSLOduration=2.690367584 podStartE2EDuration="7.300957522s" totalImagesPullingTime="3.595252914s" totalInitContainerRuntime="1.015337024s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:06:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:06:21.300104409 +0000 UTC m=+360.910857798" watchObservedRunningTime="2026-07-21 21:06:21.300957522 +0000 UTC m=+360.911710906" Jul 21 21:06:32.284681 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:32.284643 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-694dcbb678-jq7lj" Jul 21 21:06:34.661150 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.661114 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm"] Jul 21 21:06:34.669641 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.669617 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.672088 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.672065 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 21 21:06:34.672228 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.672211 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 21:06:34.673115 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.673081 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 21 21:06:34.673221 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.673128 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 21 21:06:34.673221 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.673139 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 21:06:34.673308 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.673222 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tdtgn\"" Jul 21 21:06:34.675836 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.675816 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm"] Jul 21 21:06:34.796070 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.796038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-cert\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.796207 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.796097 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-metrics-cert\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.796207 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.796135 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4q6p\" (UniqueName: \"kubernetes.io/projected/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-kube-api-access-s4q6p\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.796207 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.796154 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-manager-config\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.896843 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.896810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-manager-config\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.897026 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.896869 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-cert\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.897026 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.896920 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-metrics-cert\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.897026 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.896965 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4q6p\" (UniqueName: \"kubernetes.io/projected/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-kube-api-access-s4q6p\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.897475 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.897440 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-manager-config\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.899466 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.899435 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-cert\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.899556 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.899479 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-metrics-cert\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.906181 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.906155 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4q6p\" (UniqueName: \"kubernetes.io/projected/9f5d702c-52fa-43cb-b7aa-1ac5aa38119a-kube-api-access-s4q6p\") pod \"lws-controller-manager-8f965b796-wnvxm\" (UID: \"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a\") " pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:34.979788 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:34.979709 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:35.107130 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:35.107100 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm"] Jul 21 21:06:35.110628 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:06:35.110576 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f5d702c_52fa_43cb_b7aa_1ac5aa38119a.slice/crio-f5f564b7daa286f67266da25764dfc80b2a0de42f86daa2f494c2e9d8ac00e2b WatchSource:0}: Error finding container f5f564b7daa286f67266da25764dfc80b2a0de42f86daa2f494c2e9d8ac00e2b: Status 404 returned error can't find the container with id f5f564b7daa286f67266da25764dfc80b2a0de42f86daa2f494c2e9d8ac00e2b Jul 21 21:06:35.327767 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:35.327728 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" event={"ID":"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a","Type":"ContainerStarted","Data":"f5f564b7daa286f67266da25764dfc80b2a0de42f86daa2f494c2e9d8ac00e2b"} Jul 21 21:06:38.341507 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:38.341469 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" event={"ID":"9f5d702c-52fa-43cb-b7aa-1ac5aa38119a","Type":"ContainerStarted","Data":"b7be167b5e14e5e77a4ab1fa500a3c6d21ced3ed855154bc842cc74dbecce9a9"} Jul 21 21:06:38.341989 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:38.341630 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:38.359114 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:38.359068 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" podStartSLOduration=2.07201506 podStartE2EDuration="4.359054052s" totalImagesPullingTime="2.287038992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:06:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:06:38.357884836 +0000 UTC m=+377.968638220" watchObservedRunningTime="2026-07-21 21:06:38.359054052 +0000 UTC m=+377.969807435" Jul 21 21:06:45.128730 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.128689 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-b46444884-9ztk5"] Jul 21 21:06:45.131921 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.131893 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" Jul 21 21:06:45.141451 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.141433 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 21 21:06:45.142014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.141993 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 21:06:45.142120 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.142037 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 21:06:45.142120 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.141993 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 21 21:06:45.142399 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.142380 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-twlhs\"" Jul 21 21:06:45.151142 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.151116 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b46444884-9ztk5"] Jul 21 21:06:45.284305 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.284278 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2735e6a0-9b32-480d-9261-865f9df1174a-tmp\") pod \"kube-auth-proxy-b46444884-9ztk5\" (UID: \"2735e6a0-9b32-480d-9261-865f9df1174a\") " pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" Jul 21 21:06:45.284492 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.284344 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2735e6a0-9b32-480d-9261-865f9df1174a-tls-certs\") pod \"kube-auth-proxy-b46444884-9ztk5\" (UID: \"2735e6a0-9b32-480d-9261-865f9df1174a\") " pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" Jul 21 21:06:45.284492 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.284363 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvgwl\" (UniqueName: \"kubernetes.io/projected/2735e6a0-9b32-480d-9261-865f9df1174a-kube-api-access-wvgwl\") pod \"kube-auth-proxy-b46444884-9ztk5\" (UID: \"2735e6a0-9b32-480d-9261-865f9df1174a\") " pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" Jul 21 21:06:45.385487 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.385404 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2735e6a0-9b32-480d-9261-865f9df1174a-tls-certs\") pod \"kube-auth-proxy-b46444884-9ztk5\" (UID: \"2735e6a0-9b32-480d-9261-865f9df1174a\") " pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" Jul 21 21:06:45.385487 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.385437 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvgwl\" (UniqueName: \"kubernetes.io/projected/2735e6a0-9b32-480d-9261-865f9df1174a-kube-api-access-wvgwl\") pod \"kube-auth-proxy-b46444884-9ztk5\" (UID: \"2735e6a0-9b32-480d-9261-865f9df1174a\") " pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" Jul 21 21:06:45.385487 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.385466 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2735e6a0-9b32-480d-9261-865f9df1174a-tmp\") pod \"kube-auth-proxy-b46444884-9ztk5\" (UID: \"2735e6a0-9b32-480d-9261-865f9df1174a\") " pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" Jul 21 21:06:45.387896 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.387873 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2735e6a0-9b32-480d-9261-865f9df1174a-tmp\") pod \"kube-auth-proxy-b46444884-9ztk5\" (UID: \"2735e6a0-9b32-480d-9261-865f9df1174a\") " pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" Jul 21 21:06:45.388081 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.388063 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2735e6a0-9b32-480d-9261-865f9df1174a-tls-certs\") pod \"kube-auth-proxy-b46444884-9ztk5\" (UID: \"2735e6a0-9b32-480d-9261-865f9df1174a\") " pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" Jul 21 21:06:45.394783 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.394756 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvgwl\" (UniqueName: \"kubernetes.io/projected/2735e6a0-9b32-480d-9261-865f9df1174a-kube-api-access-wvgwl\") pod \"kube-auth-proxy-b46444884-9ztk5\" (UID: \"2735e6a0-9b32-480d-9261-865f9df1174a\") " pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" Jul 21 21:06:45.441846 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.441822 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" Jul 21 21:06:45.597632 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:45.597605 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b46444884-9ztk5"] Jul 21 21:06:45.600072 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:06:45.600050 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2735e6a0_9b32_480d_9261_865f9df1174a.slice/crio-4dfffdab87444be06f81f26b09408bd778cf3899b32a3fabd639162ec6827724 WatchSource:0}: Error finding container 4dfffdab87444be06f81f26b09408bd778cf3899b32a3fabd639162ec6827724: Status 404 returned error can't find the container with id 4dfffdab87444be06f81f26b09408bd778cf3899b32a3fabd639162ec6827724 Jul 21 21:06:46.377830 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:46.377772 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" event={"ID":"2735e6a0-9b32-480d-9261-865f9df1174a","Type":"ContainerStarted","Data":"4dfffdab87444be06f81f26b09408bd778cf3899b32a3fabd639162ec6827724"} Jul 21 21:06:49.350497 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:49.350459 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8f965b796-wnvxm" Jul 21 21:06:49.391410 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:49.391375 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" event={"ID":"2735e6a0-9b32-480d-9261-865f9df1174a","Type":"ContainerStarted","Data":"df8fcf589161df36a8b32cdeff43a7a08bae25f8c783bb8fd2d7b46b9ab46b3d"} Jul 21 21:06:49.408468 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:06:49.408419 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-b46444884-9ztk5" podStartSLOduration=1.328632395 podStartE2EDuration="4.408402982s" totalImagesPullingTime="3.079770587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:06:49.407886195 +0000 UTC m=+389.018639582" watchObservedRunningTime="2026-07-21 21:06:49.408402982 +0000 UTC m=+389.019156407" Jul 21 21:08:12.480566 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.480525 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n"] Jul 21 21:08:12.484174 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.484154 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:12.487907 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.487875 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 21 21:08:12.487907 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.487877 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 21:08:12.488103 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.487877 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 21:08:12.488103 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.487877 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 21 21:08:12.488337 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.488320 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lwr2p\"" Jul 21 21:08:12.498138 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.498114 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n"] Jul 21 21:08:12.592863 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.592831 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d7k4n\" (UID: \"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:12.593071 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.592891 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4czx5\" (UniqueName: \"kubernetes.io/projected/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-kube-api-access-4czx5\") pod \"kuadrant-console-plugin-66b5774985-d7k4n\" (UID: \"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:12.593071 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.592942 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d7k4n\" (UID: \"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:12.693728 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.693676 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d7k4n\" (UID: \"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:12.693963 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.693757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4czx5\" (UniqueName: \"kubernetes.io/projected/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-kube-api-access-4czx5\") pod \"kuadrant-console-plugin-66b5774985-d7k4n\" (UID: \"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:12.693963 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.693790 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d7k4n\" (UID: \"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:12.693963 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:08:12.693913 2568 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 21 21:08:12.694137 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:08:12.694000 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-plugin-serving-cert podName:b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051 nodeName:}" failed. No retries permitted until 2026-07-21 21:08:13.19397494 +0000 UTC m=+472.804728305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-d7k4n" (UID: "b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051") : secret "plugin-serving-cert" not found Jul 21 21:08:12.694520 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.694491 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d7k4n\" (UID: \"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:12.704390 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:12.704361 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4czx5\" (UniqueName: \"kubernetes.io/projected/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-kube-api-access-4czx5\") pod \"kuadrant-console-plugin-66b5774985-d7k4n\" (UID: \"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:13.200408 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:13.200364 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d7k4n\" (UID: \"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:13.205206 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:13.204077 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d7k4n\" (UID: \"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:13.393369 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:13.393334 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" Jul 21 21:08:13.528445 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:13.528421 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n"] Jul 21 21:08:13.531144 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:08:13.531107 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2b6cc84_ff0d_4bed_b9b9_c5bc79ea4051.slice/crio-24f4130dbd3ed8d7a94f647fc0a43a9df60c3880a551ade0c4c7b55f9a85cdf7 WatchSource:0}: Error finding container 24f4130dbd3ed8d7a94f647fc0a43a9df60c3880a551ade0c4c7b55f9a85cdf7: Status 404 returned error can't find the container with id 24f4130dbd3ed8d7a94f647fc0a43a9df60c3880a551ade0c4c7b55f9a85cdf7 Jul 21 21:08:13.716371 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:13.716340 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" event={"ID":"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051","Type":"ContainerStarted","Data":"24f4130dbd3ed8d7a94f647fc0a43a9df60c3880a551ade0c4c7b55f9a85cdf7"} Jul 21 21:08:36.832165 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:36.832127 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" event={"ID":"b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051","Type":"ContainerStarted","Data":"670ff9ee843a09e8642f7395b6856e10053e7992c2f5c1c6b3d966df93f952e1"} Jul 21 21:08:36.851146 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:36.851095 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7k4n" podStartSLOduration=2.580125724 podStartE2EDuration="24.851080746s" totalImagesPullingTime="22.270955022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:08:36.850062458 +0000 UTC m=+496.460815843" watchObservedRunningTime="2026-07-21 21:08:36.851080746 +0000 UTC m=+496.461834129" Jul 21 21:08:53.155954 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.155915 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:08:53.173675 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.173647 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:08:53.173813 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.173752 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" Jul 21 21:08:53.176407 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.176389 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 21 21:08:53.186175 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.186155 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:08:53.242742 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.242714 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8e552cae-91ec-4f23-84ba-5a2859d47050-config-file\") pod \"limitador-limitador-54d6d988d5-8t4bw\" (UID: \"8e552cae-91ec-4f23-84ba-5a2859d47050\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" Jul 21 21:08:53.242876 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.242770 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksgks\" (UniqueName: \"kubernetes.io/projected/8e552cae-91ec-4f23-84ba-5a2859d47050-kube-api-access-ksgks\") pod \"limitador-limitador-54d6d988d5-8t4bw\" (UID: \"8e552cae-91ec-4f23-84ba-5a2859d47050\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" Jul 21 21:08:53.343596 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.343563 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8e552cae-91ec-4f23-84ba-5a2859d47050-config-file\") pod \"limitador-limitador-54d6d988d5-8t4bw\" (UID: \"8e552cae-91ec-4f23-84ba-5a2859d47050\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" Jul 21 21:08:53.343770 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.343674 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksgks\" (UniqueName: \"kubernetes.io/projected/8e552cae-91ec-4f23-84ba-5a2859d47050-kube-api-access-ksgks\") pod \"limitador-limitador-54d6d988d5-8t4bw\" (UID: \"8e552cae-91ec-4f23-84ba-5a2859d47050\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" Jul 21 21:08:53.344187 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.344164 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8e552cae-91ec-4f23-84ba-5a2859d47050-config-file\") pod \"limitador-limitador-54d6d988d5-8t4bw\" (UID: \"8e552cae-91ec-4f23-84ba-5a2859d47050\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" Jul 21 21:08:53.351511 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.351487 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksgks\" (UniqueName: \"kubernetes.io/projected/8e552cae-91ec-4f23-84ba-5a2859d47050-kube-api-access-ksgks\") pod \"limitador-limitador-54d6d988d5-8t4bw\" (UID: \"8e552cae-91ec-4f23-84ba-5a2859d47050\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" Jul 21 21:08:53.485527 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.485447 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" Jul 21 21:08:53.622854 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.622831 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:08:53.625352 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:08:53.625328 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e552cae_91ec_4f23_84ba_5a2859d47050.slice/crio-e92b5249838057a6b9c109b2688304d208bc4fec1533ddee0778e37c93d135b6 WatchSource:0}: Error finding container e92b5249838057a6b9c109b2688304d208bc4fec1533ddee0778e37c93d135b6: Status 404 returned error can't find the container with id e92b5249838057a6b9c109b2688304d208bc4fec1533ddee0778e37c93d135b6 Jul 21 21:08:53.643116 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.643091 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-8shdv"] Jul 21 21:08:53.647676 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.647658 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8shdv" Jul 21 21:08:53.649936 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.649914 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nkn2q\"" Jul 21 21:08:53.653113 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.653094 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8shdv"] Jul 21 21:08:53.745929 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.745859 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rd8g\" (UniqueName: \"kubernetes.io/projected/0b0f5026-7789-42cb-b691-5578e34a40d6-kube-api-access-2rd8g\") pod \"authorino-58cd955675-8shdv\" (UID: \"0b0f5026-7789-42cb-b691-5578e34a40d6\") " pod="kuadrant-system/authorino-58cd955675-8shdv" Jul 21 21:08:53.846310 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.846279 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rd8g\" (UniqueName: \"kubernetes.io/projected/0b0f5026-7789-42cb-b691-5578e34a40d6-kube-api-access-2rd8g\") pod \"authorino-58cd955675-8shdv\" (UID: \"0b0f5026-7789-42cb-b691-5578e34a40d6\") " pod="kuadrant-system/authorino-58cd955675-8shdv" Jul 21 21:08:53.854193 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.854170 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rd8g\" (UniqueName: \"kubernetes.io/projected/0b0f5026-7789-42cb-b691-5578e34a40d6-kube-api-access-2rd8g\") pod \"authorino-58cd955675-8shdv\" (UID: \"0b0f5026-7789-42cb-b691-5578e34a40d6\") " pod="kuadrant-system/authorino-58cd955675-8shdv" Jul 21 21:08:53.899570 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.899531 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" event={"ID":"8e552cae-91ec-4f23-84ba-5a2859d47050","Type":"ContainerStarted","Data":"e92b5249838057a6b9c109b2688304d208bc4fec1533ddee0778e37c93d135b6"} Jul 21 21:08:53.957490 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:53.957461 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8shdv" Jul 21 21:08:54.114598 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:54.114518 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8shdv"] Jul 21 21:08:54.118515 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:08:54.118477 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b0f5026_7789_42cb_b691_5578e34a40d6.slice/crio-b13860165f265034047a30d09cc39204ef6310fe954dcd40619d49b6bd713fb7 WatchSource:0}: Error finding container b13860165f265034047a30d09cc39204ef6310fe954dcd40619d49b6bd713fb7: Status 404 returned error can't find the container with id b13860165f265034047a30d09cc39204ef6310fe954dcd40619d49b6bd713fb7 Jul 21 21:08:54.914574 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:54.914515 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8shdv" event={"ID":"0b0f5026-7789-42cb-b691-5578e34a40d6","Type":"ContainerStarted","Data":"b13860165f265034047a30d09cc39204ef6310fe954dcd40619d49b6bd713fb7"} Jul 21 21:08:57.914540 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:57.914505 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8shdv"] Jul 21 21:08:59.931462 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:59.931424 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" event={"ID":"8e552cae-91ec-4f23-84ba-5a2859d47050","Type":"ContainerStarted","Data":"213f189c441e23da0b66713cd44f761aefbe2d663f13e8ac1275bb2916ce53cd"} Jul 21 21:08:59.931930 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:59.931563 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" Jul 21 21:08:59.932970 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:59.932938 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8shdv" event={"ID":"0b0f5026-7789-42cb-b691-5578e34a40d6","Type":"ContainerStarted","Data":"9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0"} Jul 21 21:08:59.933086 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:59.933029 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-8shdv" podUID="0b0f5026-7789-42cb-b691-5578e34a40d6" containerName="authorino" containerID="cri-o://9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0" gracePeriod=30 Jul 21 21:08:59.947630 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:59.947560 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" podStartSLOduration=1.651300604 podStartE2EDuration="6.947524976s" totalImagesPullingTime="5.296224372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:08:59.946842394 +0000 UTC m=+519.557595806" watchObservedRunningTime="2026-07-21 21:08:59.947524976 +0000 UTC m=+519.558278356" Jul 21 21:08:59.960442 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:08:59.960405 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-8shdv" podStartSLOduration=1.829163554 podStartE2EDuration="6.96039449s" totalImagesPullingTime="5.131230936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:08:59.959844167 +0000 UTC m=+519.570597552" watchObservedRunningTime="2026-07-21 21:08:59.96039449 +0000 UTC m=+519.571147874" Jul 21 21:09:00.243216 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.243184 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8shdv" Jul 21 21:09:00.303577 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.303549 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rd8g\" (UniqueName: \"kubernetes.io/projected/0b0f5026-7789-42cb-b691-5578e34a40d6-kube-api-access-2rd8g\") pod \"0b0f5026-7789-42cb-b691-5578e34a40d6\" (UID: \"0b0f5026-7789-42cb-b691-5578e34a40d6\") " Jul 21 21:09:00.305794 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.305761 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b0f5026-7789-42cb-b691-5578e34a40d6-kube-api-access-2rd8g" (OuterVolumeSpecName: "kube-api-access-2rd8g") pod "0b0f5026-7789-42cb-b691-5578e34a40d6" (UID: "0b0f5026-7789-42cb-b691-5578e34a40d6"). InnerVolumeSpecName "kube-api-access-2rd8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:09:00.404285 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.404246 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rd8g\" (UniqueName: \"kubernetes.io/projected/0b0f5026-7789-42cb-b691-5578e34a40d6-kube-api-access-2rd8g\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:09:00.938357 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.938324 2568 generic.go:356] "Generic (PLEG): container finished" podID="0b0f5026-7789-42cb-b691-5578e34a40d6" containerID="9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0" exitCode=0 Jul 21 21:09:00.938751 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.938369 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8shdv" Jul 21 21:09:00.938751 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.938403 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8shdv" event={"ID":"0b0f5026-7789-42cb-b691-5578e34a40d6","Type":"ContainerDied","Data":"9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0"} Jul 21 21:09:00.938751 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.938448 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8shdv" event={"ID":"0b0f5026-7789-42cb-b691-5578e34a40d6","Type":"ContainerDied","Data":"b13860165f265034047a30d09cc39204ef6310fe954dcd40619d49b6bd713fb7"} Jul 21 21:09:00.938751 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.938475 2568 scope.go:117] "RemoveContainer" containerID="9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0" Jul 21 21:09:00.947673 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.947650 2568 scope.go:117] "RemoveContainer" containerID="9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0" Jul 21 21:09:00.947975 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:09:00.947944 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0\": container with ID starting with 9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0 not found: ID does not exist" containerID="9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0" Jul 21 21:09:00.948053 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.947976 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0"} err="failed to get container status \"9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0\": rpc error: code = NotFound desc = could not find container \"9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0\": container with ID starting with 9630be223e8dff0dcc67e48d03f2a3ed4a20c0bd3af688e8dfe7270a435d13c0 not found: ID does not exist" Jul 21 21:09:00.954272 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.954249 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8shdv"] Jul 21 21:09:00.958086 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:00.958066 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-8shdv"] Jul 21 21:09:02.912435 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:02.912399 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b0f5026-7789-42cb-b691-5578e34a40d6" path="/var/lib/kubelet/pods/0b0f5026-7789-42cb-b691-5578e34a40d6/volumes" Jul 21 21:09:10.939926 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:10.939895 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8t4bw" Jul 21 21:09:27.673027 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:27.672987 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n9gw5"] Jul 21 21:09:27.673425 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:27.673288 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b0f5026-7789-42cb-b691-5578e34a40d6" containerName="authorino" Jul 21 21:09:27.673425 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:27.673300 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b0f5026-7789-42cb-b691-5578e34a40d6" containerName="authorino" Jul 21 21:09:27.673425 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:27.673351 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b0f5026-7789-42cb-b691-5578e34a40d6" containerName="authorino" Jul 21 21:09:27.756510 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:27.756480 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n9gw5"] Jul 21 21:09:27.756676 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:27.756581 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n9gw5" Jul 21 21:09:27.758966 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:27.758946 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nkn2q\"" Jul 21 21:09:27.824764 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:27.824734 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8cnf\" (UniqueName: \"kubernetes.io/projected/e21a3549-33dd-4c68-8fbd-d45672392d69-kube-api-access-h8cnf\") pod \"authorino-5d4b67c446-n9gw5\" (UID: \"e21a3549-33dd-4c68-8fbd-d45672392d69\") " pod="kuadrant-system/authorino-5d4b67c446-n9gw5" Jul 21 21:09:27.925203 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:27.925137 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8cnf\" (UniqueName: \"kubernetes.io/projected/e21a3549-33dd-4c68-8fbd-d45672392d69-kube-api-access-h8cnf\") pod \"authorino-5d4b67c446-n9gw5\" (UID: \"e21a3549-33dd-4c68-8fbd-d45672392d69\") " pod="kuadrant-system/authorino-5d4b67c446-n9gw5" Jul 21 21:09:27.933580 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:27.933556 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n9gw5"] Jul 21 21:09:27.933793 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:09:27.933773 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-h8cnf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-n9gw5" podUID="e21a3549-33dd-4c68-8fbd-d45672392d69" Jul 21 21:09:27.936360 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:27.936341 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8cnf\" (UniqueName: \"kubernetes.io/projected/e21a3549-33dd-4c68-8fbd-d45672392d69-kube-api-access-h8cnf\") pod \"authorino-5d4b67c446-n9gw5\" (UID: \"e21a3549-33dd-4c68-8fbd-d45672392d69\") " pod="kuadrant-system/authorino-5d4b67c446-n9gw5" Jul 21 21:09:28.047601 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:28.047564 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n9gw5" Jul 21 21:09:28.077342 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:28.077320 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n9gw5" Jul 21 21:09:28.227627 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:28.227542 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8cnf\" (UniqueName: \"kubernetes.io/projected/e21a3549-33dd-4c68-8fbd-d45672392d69-kube-api-access-h8cnf\") pod \"e21a3549-33dd-4c68-8fbd-d45672392d69\" (UID: \"e21a3549-33dd-4c68-8fbd-d45672392d69\") " Jul 21 21:09:28.229771 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:28.229742 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e21a3549-33dd-4c68-8fbd-d45672392d69-kube-api-access-h8cnf" (OuterVolumeSpecName: "kube-api-access-h8cnf") pod "e21a3549-33dd-4c68-8fbd-d45672392d69" (UID: "e21a3549-33dd-4c68-8fbd-d45672392d69"). InnerVolumeSpecName "kube-api-access-h8cnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:09:28.328078 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:28.328047 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8cnf\" (UniqueName: \"kubernetes.io/projected/e21a3549-33dd-4c68-8fbd-d45672392d69-kube-api-access-h8cnf\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:09:29.052253 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:29.052228 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n9gw5" Jul 21 21:09:29.080717 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:29.080688 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n9gw5"] Jul 21 21:09:29.084727 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:29.084703 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n9gw5"] Jul 21 21:09:30.911601 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:09:30.911550 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e21a3549-33dd-4c68-8fbd-d45672392d69" path="/var/lib/kubelet/pods/e21a3549-33dd-4c68-8fbd-d45672392d69/volumes" Jul 21 21:10:11.608148 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:10:11.608099 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:10:20.841891 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:10:20.841863 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:10:20.846757 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:10:20.846728 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:10:20.850598 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:10:20.850559 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:10:20.855356 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:10:20.855336 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:10:44.417572 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:10:44.417494 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:10:50.395460 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:10:50.395423 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:11:00.195570 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:00.195535 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:11:20.797073 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:20.797041 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:11:31.228791 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.228757 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj"] Jul 21 21:11:31.232466 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.232441 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.235305 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.235278 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 21:11:31.235444 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.235311 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 21:11:31.236249 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.236227 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-t48v6\"" Jul 21 21:11:31.236385 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.236272 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 21 21:11:31.244468 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.244444 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj"] Jul 21 21:11:31.345608 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.345565 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.345770 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.345648 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.345770 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.345690 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.345770 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.345715 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vn77\" (UniqueName: \"kubernetes.io/projected/7f74a56d-3b37-43c7-9872-bd6554ba95e6-kube-api-access-2vn77\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.345876 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.345785 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.345876 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.345832 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f74a56d-3b37-43c7-9872-bd6554ba95e6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.447036 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.447004 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.447181 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.447048 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.447181 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.447072 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.447181 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.447088 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vn77\" (UniqueName: \"kubernetes.io/projected/7f74a56d-3b37-43c7-9872-bd6554ba95e6-kube-api-access-2vn77\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.447323 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.447215 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.447323 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.447289 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f74a56d-3b37-43c7-9872-bd6554ba95e6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.447516 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.447495 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.447576 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.447553 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.447576 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.447563 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.449368 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.449344 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f74a56d-3b37-43c7-9872-bd6554ba95e6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.449761 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.449744 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f74a56d-3b37-43c7-9872-bd6554ba95e6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.456645 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.456626 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vn77\" (UniqueName: \"kubernetes.io/projected/7f74a56d-3b37-43c7-9872-bd6554ba95e6-kube-api-access-2vn77\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ndwkj\" (UID: \"7f74a56d-3b37-43c7-9872-bd6554ba95e6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.545242 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.545218 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:31.679353 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.679323 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj"] Jul 21 21:11:31.682522 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:11:31.682491 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f74a56d_3b37_43c7_9872_bd6554ba95e6.slice/crio-091d83c5ee5c533505d6760dc4da4f9a4ef9c0108947fd74b8ef1e605da2b998 WatchSource:0}: Error finding container 091d83c5ee5c533505d6760dc4da4f9a4ef9c0108947fd74b8ef1e605da2b998: Status 404 returned error can't find the container with id 091d83c5ee5c533505d6760dc4da4f9a4ef9c0108947fd74b8ef1e605da2b998 Jul 21 21:11:31.683923 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:31.683904 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 21:11:32.577406 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:32.577369 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" event={"ID":"7f74a56d-3b37-43c7-9872-bd6554ba95e6","Type":"ContainerStarted","Data":"091d83c5ee5c533505d6760dc4da4f9a4ef9c0108947fd74b8ef1e605da2b998"} Jul 21 21:11:33.197670 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:33.197636 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:11:33.582604 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:33.582556 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" event={"ID":"7f74a56d-3b37-43c7-9872-bd6554ba95e6","Type":"ContainerStarted","Data":"5858a94cbb63d7c63e52bd59444b2902b382fd08c0cbd735912d022619f95d8c"} Jul 21 21:11:33.601502 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:33.601455 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" podStartSLOduration=1.522643229 podStartE2EDuration="2.601439757s" totalImagesPullingTime="1.078796528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:11:33.599758798 +0000 UTC m=+673.210512181" watchObservedRunningTime="2026-07-21 21:11:33.601439757 +0000 UTC m=+673.212193140" Jul 21 21:11:39.729064 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:39.729033 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt"] Jul 21 21:11:39.732458 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:39.732437 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:39.734874 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:39.734856 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 21 21:11:39.740276 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:39.740158 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt"] Jul 21 21:11:39.914470 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:39.914437 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9flfq\" (UniqueName: \"kubernetes.io/projected/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-kube-api-access-9flfq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:39.914697 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:39.914479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:39.914697 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:39.914504 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:39.914697 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:39.914572 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:39.914697 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:39.914640 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:39.914697 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:39.914693 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.015754 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.015672 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9flfq\" (UniqueName: \"kubernetes.io/projected/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-kube-api-access-9flfq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.015754 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.015721 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.015754 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.015750 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.016073 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.015780 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.016073 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.015800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.016073 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.015831 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.016264 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.016236 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.016378 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.016284 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.016378 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.016333 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.018215 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.018198 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.018460 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.018444 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.023682 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.023664 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9flfq\" (UniqueName: \"kubernetes.io/projected/f75efee8-7d35-44f1-9ad1-e6d1d0737c32-kube-api-access-9flfq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt\" (UID: \"f75efee8-7d35-44f1-9ad1-e6d1d0737c32\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.042670 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.042646 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:40.171627 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.171574 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt"] Jul 21 21:11:40.174355 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:11:40.174329 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf75efee8_7d35_44f1_9ad1_e6d1d0737c32.slice/crio-f97d02cb5c021c4de34eb2006d896036df60f080b1888ffef365a6e0d1c2d2b6 WatchSource:0}: Error finding container f97d02cb5c021c4de34eb2006d896036df60f080b1888ffef365a6e0d1c2d2b6: Status 404 returned error can't find the container with id f97d02cb5c021c4de34eb2006d896036df60f080b1888ffef365a6e0d1c2d2b6 Jul 21 21:11:40.613747 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.613712 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" event={"ID":"f75efee8-7d35-44f1-9ad1-e6d1d0737c32","Type":"ContainerStarted","Data":"b72c389eaf412366f51d755b587db512310f1043cb55fce5233475b1dbea025c"} Jul 21 21:11:40.613882 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.613758 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" event={"ID":"f75efee8-7d35-44f1-9ad1-e6d1d0737c32","Type":"ContainerStarted","Data":"f97d02cb5c021c4de34eb2006d896036df60f080b1888ffef365a6e0d1c2d2b6"} Jul 21 21:11:40.629962 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:40.629917 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" podStartSLOduration=1.3961394010000001 podStartE2EDuration="1.629902674s" totalImagesPullingTime="233.763273ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:11:40.629467728 +0000 UTC m=+680.240221116" watchObservedRunningTime="2026-07-21 21:11:40.629902674 +0000 UTC m=+680.240656069" Jul 21 21:11:41.545467 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:41.545433 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:41.545848 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:41.545493 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:41.557809 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:41.557787 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:41.628737 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:41.628712 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ndwkj" Jul 21 21:11:43.102986 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:43.102952 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:11:50.043562 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:50.043533 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:50.044128 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:50.043628 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:50.056019 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:50.055999 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:11:50.667875 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:11:50.667847 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt" Jul 21 21:12:46.767497 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:46.767458 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54bdc969b5-8jjtb"] Jul 21 21:12:46.771923 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:46.771887 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54bdc969b5-8jjtb" Jul 21 21:12:46.775529 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:46.775482 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nkn2q\"" Jul 21 21:12:46.775529 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:46.775506 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 21 21:12:46.777852 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:46.777833 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54bdc969b5-8jjtb"] Jul 21 21:12:46.850408 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:46.850385 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/85d8197e-aeb4-4cdd-8084-fa65ef54d317-tls-cert\") pod \"authorino-54bdc969b5-8jjtb\" (UID: \"85d8197e-aeb4-4cdd-8084-fa65ef54d317\") " pod="kuadrant-system/authorino-54bdc969b5-8jjtb" Jul 21 21:12:46.850533 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:46.850434 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swhjg\" (UniqueName: \"kubernetes.io/projected/85d8197e-aeb4-4cdd-8084-fa65ef54d317-kube-api-access-swhjg\") pod \"authorino-54bdc969b5-8jjtb\" (UID: \"85d8197e-aeb4-4cdd-8084-fa65ef54d317\") " pod="kuadrant-system/authorino-54bdc969b5-8jjtb" Jul 21 21:12:46.951371 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:46.951343 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/85d8197e-aeb4-4cdd-8084-fa65ef54d317-tls-cert\") pod \"authorino-54bdc969b5-8jjtb\" (UID: \"85d8197e-aeb4-4cdd-8084-fa65ef54d317\") " pod="kuadrant-system/authorino-54bdc969b5-8jjtb" Jul 21 21:12:46.951484 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:46.951392 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swhjg\" (UniqueName: \"kubernetes.io/projected/85d8197e-aeb4-4cdd-8084-fa65ef54d317-kube-api-access-swhjg\") pod \"authorino-54bdc969b5-8jjtb\" (UID: \"85d8197e-aeb4-4cdd-8084-fa65ef54d317\") " pod="kuadrant-system/authorino-54bdc969b5-8jjtb" Jul 21 21:12:46.953886 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:46.953859 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/85d8197e-aeb4-4cdd-8084-fa65ef54d317-tls-cert\") pod \"authorino-54bdc969b5-8jjtb\" (UID: \"85d8197e-aeb4-4cdd-8084-fa65ef54d317\") " pod="kuadrant-system/authorino-54bdc969b5-8jjtb" Jul 21 21:12:46.959790 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:46.959769 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swhjg\" (UniqueName: \"kubernetes.io/projected/85d8197e-aeb4-4cdd-8084-fa65ef54d317-kube-api-access-swhjg\") pod \"authorino-54bdc969b5-8jjtb\" (UID: \"85d8197e-aeb4-4cdd-8084-fa65ef54d317\") " pod="kuadrant-system/authorino-54bdc969b5-8jjtb" Jul 21 21:12:47.082347 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:47.082325 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54bdc969b5-8jjtb" Jul 21 21:12:47.213978 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:47.213956 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54bdc969b5-8jjtb"] Jul 21 21:12:47.216339 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:12:47.216310 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85d8197e_aeb4_4cdd_8084_fa65ef54d317.slice/crio-03bf7f07fe74680e21537dcf242da34ce3a2bc3dd5e5f1b1646864f8351b80de WatchSource:0}: Error finding container 03bf7f07fe74680e21537dcf242da34ce3a2bc3dd5e5f1b1646864f8351b80de: Status 404 returned error can't find the container with id 03bf7f07fe74680e21537dcf242da34ce3a2bc3dd5e5f1b1646864f8351b80de Jul 21 21:12:47.899336 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:47.899299 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54bdc969b5-8jjtb" event={"ID":"85d8197e-aeb4-4cdd-8084-fa65ef54d317","Type":"ContainerStarted","Data":"51486b28d67c7c13fddc5cd826fbdb1815aff0a22327b0bbef9887ecd0f425d8"} Jul 21 21:12:47.899728 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:47.899342 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54bdc969b5-8jjtb" event={"ID":"85d8197e-aeb4-4cdd-8084-fa65ef54d317","Type":"ContainerStarted","Data":"03bf7f07fe74680e21537dcf242da34ce3a2bc3dd5e5f1b1646864f8351b80de"} Jul 21 21:12:47.919235 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:12:47.919187 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54bdc969b5-8jjtb" podStartSLOduration=1.426281325 podStartE2EDuration="1.919173294s" totalImagesPullingTime="492.891969ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:12:47.917445078 +0000 UTC m=+747.528198463" watchObservedRunningTime="2026-07-21 21:12:47.919173294 +0000 UTC m=+747.529926678" Jul 21 21:13:17.731253 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:13:17.731220 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:13:28.011424 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:13:28.011389 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:13:39.247649 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:13:39.247612 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:15:20.871021 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:15:20.870988 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:15:20.876552 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:15:20.876518 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:15:20.880200 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:15:20.880174 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:15:20.889243 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:15:20.889227 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:15:23.106957 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:15:23.106921 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:15:30.098331 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:15:30.098298 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:16:02.997103 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:16:02.997067 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:16:12.590978 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:16:12.590932 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:16:21.306962 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:16:21.306926 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:16:30.398321 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:16:30.398284 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:16:39.695868 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:16:39.695834 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:16:51.196151 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:16:51.196078 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:17:01.806617 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:17:01.806566 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:17:11.998024 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:17:11.997992 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:17:34.896562 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:17:34.896525 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:17:46.501327 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:17:46.501288 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:18:09.408937 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:18:09.408898 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:18:21.699475 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:18:21.699441 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:18:36.699826 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:18:36.699780 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:18:46.106495 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:18:46.106461 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:18:55.803757 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:18:55.803719 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:19:05.193636 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:19:05.193597 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:19:14.201836 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:19:14.201801 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:20:20.904246 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:20:20.904216 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:20:20.909541 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:20:20.909515 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:20:20.912122 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:20:20.912102 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:20:20.916828 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:20:20.916810 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:21:02.799559 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:21:02.799520 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:21:11.801288 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:21:11.801208 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:21:24.706955 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:21:24.706916 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:21:35.995915 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:21:35.995876 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:21:47.499211 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:21:47.499173 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:21:57.706642 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:21:57.706605 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:22:06.590263 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:22:06.590229 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:22:17.498021 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:22:17.497984 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:22:29.395546 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:22:29.395515 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:22:37.692804 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:22:37.692764 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:22:49.392970 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:22:49.392891 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:23:10.799751 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:10.799716 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:23:20.400966 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:20.400932 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:23:32.728363 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.728331 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5"] Jul 21 21:23:32.731817 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.731797 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.734320 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.734299 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-d178517c-kserve-self-signed-certs\"" Jul 21 21:23:32.739750 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.739727 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5"] Jul 21 21:23:32.756937 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.756918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tmp-dir\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.757046 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.756947 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-dshm\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.757046 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.756965 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tls-certs\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.757046 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.756988 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4msf\" (UniqueName: \"kubernetes.io/projected/9b24b7ee-6248-4d33-ad9e-44b894e9c011-kube-api-access-m4msf\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.757208 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.757142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-model-cache\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.757208 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.757186 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-home\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.858376 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.858348 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tmp-dir\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.858492 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.858383 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-dshm\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.858492 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.858399 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tls-certs\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.858492 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.858422 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4msf\" (UniqueName: \"kubernetes.io/projected/9b24b7ee-6248-4d33-ad9e-44b894e9c011-kube-api-access-m4msf\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.858492 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.858458 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-model-cache\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.858492 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.858483 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-home\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.858828 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.858805 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tmp-dir\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.858921 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.858898 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-model-cache\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.858964 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.858924 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-home\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.860887 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.860869 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-dshm\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.861247 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.861226 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tls-certs\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:32.866669 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:32.866649 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4msf\" (UniqueName: \"kubernetes.io/projected/9b24b7ee-6248-4d33-ad9e-44b894e9c011-kube-api-access-m4msf\") pod \"e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:33.041170 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:33.041086 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:33.167401 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:33.167377 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5"] Jul 21 21:23:33.169341 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:23:33.169310 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b24b7ee_6248_4d33_ad9e_44b894e9c011.slice/crio-80b51013145a68c6a2b2f7888fa92dc1b07004b66ba88f3794b7e797b36578ca WatchSource:0}: Error finding container 80b51013145a68c6a2b2f7888fa92dc1b07004b66ba88f3794b7e797b36578ca: Status 404 returned error can't find the container with id 80b51013145a68c6a2b2f7888fa92dc1b07004b66ba88f3794b7e797b36578ca Jul 21 21:23:33.676899 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:33.676864 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" event={"ID":"9b24b7ee-6248-4d33-ad9e-44b894e9c011","Type":"ContainerStarted","Data":"5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9"} Jul 21 21:23:33.677063 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:33.676907 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" event={"ID":"9b24b7ee-6248-4d33-ad9e-44b894e9c011","Type":"ContainerStarted","Data":"80b51013145a68c6a2b2f7888fa92dc1b07004b66ba88f3794b7e797b36578ca"} Jul 21 21:23:33.694175 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:33.694134 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" podStartSLOduration=1.69411867 podStartE2EDuration="1.69411867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:23:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:23:33.692493866 +0000 UTC m=+1393.303247250" watchObservedRunningTime="2026-07-21 21:23:33.69411867 +0000 UTC m=+1393.304872056" Jul 21 21:23:35.001793 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:35.001755 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:23:43.041778 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:43.041736 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:43.041778 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:43.041790 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:43.054157 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:43.054130 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:43.733532 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:43.733505 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:23:54.172696 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:54.172655 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5"] Jul 21 21:23:57.103253 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:23:57.103221 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:24:09.197344 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.197280 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" podUID="9b24b7ee-6248-4d33-ad9e-44b894e9c011" containerName="main" containerID="cri-o://5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9" gracePeriod=45 Jul 21 21:24:09.437380 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.437359 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:24:09.564832 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.564806 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tls-certs\") pod \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " Jul 21 21:24:09.564960 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.564846 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4msf\" (UniqueName: \"kubernetes.io/projected/9b24b7ee-6248-4d33-ad9e-44b894e9c011-kube-api-access-m4msf\") pod \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " Jul 21 21:24:09.564960 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.564881 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-dshm\") pod \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " Jul 21 21:24:09.564960 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.564945 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-model-cache\") pod \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " Jul 21 21:24:09.565110 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.564968 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-home\") pod \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " Jul 21 21:24:09.565110 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.564993 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tmp-dir\") pod \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\" (UID: \"9b24b7ee-6248-4d33-ad9e-44b894e9c011\") " Jul 21 21:24:09.565330 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.565274 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-model-cache" (OuterVolumeSpecName: "model-cache") pod "9b24b7ee-6248-4d33-ad9e-44b894e9c011" (UID: "9b24b7ee-6248-4d33-ad9e-44b894e9c011"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:24:09.565444 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.565281 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-home" (OuterVolumeSpecName: "home") pod "9b24b7ee-6248-4d33-ad9e-44b894e9c011" (UID: "9b24b7ee-6248-4d33-ad9e-44b894e9c011"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:24:09.565444 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.565346 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9b24b7ee-6248-4d33-ad9e-44b894e9c011" (UID: "9b24b7ee-6248-4d33-ad9e-44b894e9c011"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:24:09.567233 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.567207 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-dshm" (OuterVolumeSpecName: "dshm") pod "9b24b7ee-6248-4d33-ad9e-44b894e9c011" (UID: "9b24b7ee-6248-4d33-ad9e-44b894e9c011"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:24:09.567313 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.567243 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b24b7ee-6248-4d33-ad9e-44b894e9c011-kube-api-access-m4msf" (OuterVolumeSpecName: "kube-api-access-m4msf") pod "9b24b7ee-6248-4d33-ad9e-44b894e9c011" (UID: "9b24b7ee-6248-4d33-ad9e-44b894e9c011"). InnerVolumeSpecName "kube-api-access-m4msf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:24:09.567313 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.567300 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9b24b7ee-6248-4d33-ad9e-44b894e9c011" (UID: "9b24b7ee-6248-4d33-ad9e-44b894e9c011"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:24:09.666218 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.666191 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-model-cache\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:24:09.666218 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.666212 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-home\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:24:09.666218 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.666221 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tmp-dir\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:24:09.666400 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.666228 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b24b7ee-6248-4d33-ad9e-44b894e9c011-tls-certs\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:24:09.666400 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.666236 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4msf\" (UniqueName: \"kubernetes.io/projected/9b24b7ee-6248-4d33-ad9e-44b894e9c011-kube-api-access-m4msf\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:24:09.666400 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.666244 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b24b7ee-6248-4d33-ad9e-44b894e9c011-dshm\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:24:09.831371 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.831294 2568 generic.go:356] "Generic (PLEG): container finished" podID="9b24b7ee-6248-4d33-ad9e-44b894e9c011" containerID="5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9" exitCode=0 Jul 21 21:24:09.831371 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.831347 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" Jul 21 21:24:09.831621 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.831370 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" event={"ID":"9b24b7ee-6248-4d33-ad9e-44b894e9c011","Type":"ContainerDied","Data":"5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9"} Jul 21 21:24:09.831621 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.831412 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5" event={"ID":"9b24b7ee-6248-4d33-ad9e-44b894e9c011","Type":"ContainerDied","Data":"80b51013145a68c6a2b2f7888fa92dc1b07004b66ba88f3794b7e797b36578ca"} Jul 21 21:24:09.831621 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.831431 2568 scope.go:117] "RemoveContainer" containerID="5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9" Jul 21 21:24:09.843563 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.843405 2568 scope.go:117] "RemoveContainer" containerID="5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9" Jul 21 21:24:09.843891 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:24:09.843865 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9\": container with ID starting with 5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9 not found: ID does not exist" containerID="5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9" Jul 21 21:24:09.843987 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.843903 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9"} err="failed to get container status \"5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9\": rpc error: code = NotFound desc = could not find container \"5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9\": container with ID starting with 5ad002e44698383fd95d1da27da5102a6ae1f07b72eeefbdbef861ed1d80eac9 not found: ID does not exist" Jul 21 21:24:09.857063 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.857035 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5"] Jul 21 21:24:09.863600 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:09.863568 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-d178517c-kserve-6b98f68759-w2jz5"] Jul 21 21:24:10.589959 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:10.589930 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:24:10.911473 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:10.911383 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b24b7ee-6248-4d33-ad9e-44b894e9c011" path="/var/lib/kubelet/pods/9b24b7ee-6248-4d33-ad9e-44b894e9c011/volumes" Jul 21 21:24:17.009932 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:17.009899 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:24:31.504907 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:31.504867 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:24:42.002178 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:42.002140 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:24:51.897989 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:24:51.897950 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:25:01.298070 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:25:01.298036 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:25:12.149269 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:25:12.149232 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:25:20.932501 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:25:20.932466 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:25:20.936997 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:25:20.936977 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:25:20.937392 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:25:20.937374 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:25:20.941910 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:25:20.941891 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:25:21.523534 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:25:21.523503 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:25:33.608912 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:25:33.608877 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:25:42.793490 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:25:42.793409 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:25:54.198230 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:25:54.198191 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:26:03.690434 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:26:03.690403 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:26:16.100930 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:26:16.100894 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:26:24.008172 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:26:24.008131 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:26:36.394623 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:26:36.394570 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:26:48.998456 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:26:48.998411 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:27:01.792716 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:27:01.792680 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:27:10.696867 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:27:10.696828 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:27:33.297569 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:27:33.297533 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:27:46.391910 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:27:46.391876 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:28:18.510370 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:28:18.510326 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:28:28.098461 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:28:28.098417 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:29:03.699624 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:29:03.699572 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:29:15.104122 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:29:15.104084 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:29:47.195286 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:29:47.195241 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:29:56.099177 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:29:56.099136 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:30:20.962297 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:30:20.962263 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:30:20.963115 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:30:20.963083 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:30:20.967375 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:30:20.967344 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:30:20.967779 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:30:20.967756 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:30:31.895318 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:30:31.895275 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:31:59.696127 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:31:59.696091 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:32:08.796922 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:32:08.796878 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:32:20.707721 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:32:20.707683 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:32:40.221624 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:32:40.221565 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:33:19.223170 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:19.223129 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:33:27.458149 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.458111 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47"] Jul 21 21:33:27.458524 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.458447 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b24b7ee-6248-4d33-ad9e-44b894e9c011" containerName="main" Jul 21 21:33:27.458524 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.458463 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b24b7ee-6248-4d33-ad9e-44b894e9c011" containerName="main" Jul 21 21:33:27.458673 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.458563 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b24b7ee-6248-4d33-ad9e-44b894e9c011" containerName="main" Jul 21 21:33:27.461399 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.461382 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.465359 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.465330 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ebe935a0\"/\"kube-root-ca.crt\"" Jul 21 21:33:27.465490 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.465475 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ebe935a0\"/\"default-dockercfg-b9xxf\"" Jul 21 21:33:27.465560 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.465491 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ebe935a0\"/\"openshift-service-ca.crt\"" Jul 21 21:33:27.465560 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.465512 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ebe935a0\"/\"e2e-lifecycle-model-ebe935a0-kserve-self-signed-certs\"" Jul 21 21:33:27.474936 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.474913 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47"] Jul 21 21:33:27.636028 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.635994 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-tmp-dir\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.636188 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.636056 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh4fj\" (UniqueName: \"kubernetes.io/projected/3142f001-7df1-4e72-9914-ea1044bad1e2-kube-api-access-nh4fj\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.636188 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.636107 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-dshm\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.636283 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.636185 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3142f001-7df1-4e72-9914-ea1044bad1e2-tls-certs\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.636283 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.636239 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-home\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.636283 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.636262 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-model-cache\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.737368 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.737290 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-dshm\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.737368 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.737345 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3142f001-7df1-4e72-9914-ea1044bad1e2-tls-certs\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.737873 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.737389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-home\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.737873 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.737408 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-model-cache\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.737873 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.737436 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-tmp-dir\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.737873 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.737618 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh4fj\" (UniqueName: \"kubernetes.io/projected/3142f001-7df1-4e72-9914-ea1044bad1e2-kube-api-access-nh4fj\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.737873 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.737839 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-home\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.737873 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.737854 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-tmp-dir\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.738357 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.737911 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-model-cache\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.739808 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.739785 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-dshm\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.740031 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.740015 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3142f001-7df1-4e72-9914-ea1044bad1e2-tls-certs\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.747812 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.747790 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh4fj\" (UniqueName: \"kubernetes.io/projected/3142f001-7df1-4e72-9914-ea1044bad1e2-kube-api-access-nh4fj\") pod \"e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.771148 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.771130 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:27.902474 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:27.902419 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47"] Jul 21 21:33:27.904742 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:33:27.904706 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3142f001_7df1_4e72_9914_ea1044bad1e2.slice/crio-c13411c13dc9a7de84f4e67e65700b2254e7dab15fbccb33650fd5d4d5dcb464 WatchSource:0}: Error finding container c13411c13dc9a7de84f4e67e65700b2254e7dab15fbccb33650fd5d4d5dcb464: Status 404 returned error can't find the container with id c13411c13dc9a7de84f4e67e65700b2254e7dab15fbccb33650fd5d4d5dcb464 Jul 21 21:33:28.315368 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:28.315323 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" event={"ID":"3142f001-7df1-4e72-9914-ea1044bad1e2","Type":"ContainerStarted","Data":"dd62ed7bd3502283eed55171e4ebe84bfc75188c3f92cfa5642b3f5e61103429"} Jul 21 21:33:28.315368 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:28.315373 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" event={"ID":"3142f001-7df1-4e72-9914-ea1044bad1e2","Type":"ContainerStarted","Data":"c13411c13dc9a7de84f4e67e65700b2254e7dab15fbccb33650fd5d4d5dcb464"} Jul 21 21:33:28.334421 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:28.334374 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" podStartSLOduration=1.334359256 podStartE2EDuration="1.334359256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:33:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:33:28.333226029 +0000 UTC m=+1987.943979414" watchObservedRunningTime="2026-07-21 21:33:28.334359256 +0000 UTC m=+1987.945112640" Jul 21 21:33:33.204774 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:33.204741 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:33:37.772078 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:37.772039 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:37.772440 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:37.772097 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:37.784523 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:37.784494 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:38.369504 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:38.369476 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:33:46.791190 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:33:46.791151 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:34:04.196862 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:04.196817 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:34:12.792126 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:12.792090 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:34:17.331798 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:17.331762 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47"] Jul 21 21:34:32.352313 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.352250 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" podUID="3142f001-7df1-4e72-9914-ea1044bad1e2" containerName="main" containerID="cri-o://dd62ed7bd3502283eed55171e4ebe84bfc75188c3f92cfa5642b3f5e61103429" gracePeriod=45 Jul 21 21:34:32.593316 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.593284 2568 generic.go:356] "Generic (PLEG): container finished" podID="3142f001-7df1-4e72-9914-ea1044bad1e2" containerID="dd62ed7bd3502283eed55171e4ebe84bfc75188c3f92cfa5642b3f5e61103429" exitCode=0 Jul 21 21:34:32.593419 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.593350 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" event={"ID":"3142f001-7df1-4e72-9914-ea1044bad1e2","Type":"ContainerDied","Data":"dd62ed7bd3502283eed55171e4ebe84bfc75188c3f92cfa5642b3f5e61103429"} Jul 21 21:34:32.593419 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.593391 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" event={"ID":"3142f001-7df1-4e72-9914-ea1044bad1e2","Type":"ContainerDied","Data":"c13411c13dc9a7de84f4e67e65700b2254e7dab15fbccb33650fd5d4d5dcb464"} Jul 21 21:34:32.593419 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.593403 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c13411c13dc9a7de84f4e67e65700b2254e7dab15fbccb33650fd5d4d5dcb464" Jul 21 21:34:32.600189 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.600173 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:34:32.741519 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.741432 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-home\") pod \"3142f001-7df1-4e72-9914-ea1044bad1e2\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " Jul 21 21:34:32.741519 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.741473 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-model-cache\") pod \"3142f001-7df1-4e72-9914-ea1044bad1e2\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " Jul 21 21:34:32.741795 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.741525 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-dshm\") pod \"3142f001-7df1-4e72-9914-ea1044bad1e2\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " Jul 21 21:34:32.741795 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.741560 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh4fj\" (UniqueName: \"kubernetes.io/projected/3142f001-7df1-4e72-9914-ea1044bad1e2-kube-api-access-nh4fj\") pod \"3142f001-7df1-4e72-9914-ea1044bad1e2\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " Jul 21 21:34:32.741795 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.741606 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3142f001-7df1-4e72-9914-ea1044bad1e2-tls-certs\") pod \"3142f001-7df1-4e72-9914-ea1044bad1e2\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " Jul 21 21:34:32.741795 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.741649 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-tmp-dir\") pod \"3142f001-7df1-4e72-9914-ea1044bad1e2\" (UID: \"3142f001-7df1-4e72-9914-ea1044bad1e2\") " Jul 21 21:34:32.741795 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.741710 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-home" (OuterVolumeSpecName: "home") pod "3142f001-7df1-4e72-9914-ea1044bad1e2" (UID: "3142f001-7df1-4e72-9914-ea1044bad1e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:34:32.742028 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.741828 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "3142f001-7df1-4e72-9914-ea1044bad1e2" (UID: "3142f001-7df1-4e72-9914-ea1044bad1e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:34:32.742028 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.741938 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3142f001-7df1-4e72-9914-ea1044bad1e2" (UID: "3142f001-7df1-4e72-9914-ea1044bad1e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:34:32.742028 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.741957 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-home\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:34:32.742028 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.741977 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-model-cache\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:34:32.744045 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.744020 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3142f001-7df1-4e72-9914-ea1044bad1e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3142f001-7df1-4e72-9914-ea1044bad1e2" (UID: "3142f001-7df1-4e72-9914-ea1044bad1e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:34:32.744154 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.744023 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-dshm" (OuterVolumeSpecName: "dshm") pod "3142f001-7df1-4e72-9914-ea1044bad1e2" (UID: "3142f001-7df1-4e72-9914-ea1044bad1e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:34:32.744154 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.744055 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3142f001-7df1-4e72-9914-ea1044bad1e2-kube-api-access-nh4fj" (OuterVolumeSpecName: "kube-api-access-nh4fj") pod "3142f001-7df1-4e72-9914-ea1044bad1e2" (UID: "3142f001-7df1-4e72-9914-ea1044bad1e2"). InnerVolumeSpecName "kube-api-access-nh4fj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:34:32.842987 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.842963 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-tmp-dir\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:34:32.842987 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.842988 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3142f001-7df1-4e72-9914-ea1044bad1e2-dshm\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:34:32.843169 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.842997 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh4fj\" (UniqueName: \"kubernetes.io/projected/3142f001-7df1-4e72-9914-ea1044bad1e2-kube-api-access-nh4fj\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:34:32.843169 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:32.843007 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3142f001-7df1-4e72-9914-ea1044bad1e2-tls-certs\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:34:33.598029 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:33.597999 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47" Jul 21 21:34:33.628613 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:33.628572 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47"] Jul 21 21:34:33.634834 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:33.634811 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ebe935a0/e2e-lifecycle-model-ebe935a0-kserve-856c5876cf-hhb47"] Jul 21 21:34:34.911694 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:34:34.911661 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3142f001-7df1-4e72-9914-ea1044bad1e2" path="/var/lib/kubelet/pods/3142f001-7df1-4e72-9914-ea1044bad1e2/volumes" Jul 21 21:35:04.409506 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:35:04.409470 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:35:14.724822 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:35:14.724786 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:35:20.987994 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:35:20.987965 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:35:20.989502 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:35:20.989479 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:35:20.992526 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:35:20.992491 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:35:20.993937 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:35:20.993916 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:36:06.798271 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:06.798233 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:36:18.102057 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.099394 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:36:18.527023 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.526941 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw"] Jul 21 21:36:18.527264 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.527252 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3142f001-7df1-4e72-9914-ea1044bad1e2" containerName="main" Jul 21 21:36:18.527264 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.527264 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3142f001-7df1-4e72-9914-ea1044bad1e2" containerName="main" Jul 21 21:36:18.527385 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.527321 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3142f001-7df1-4e72-9914-ea1044bad1e2" containerName="main" Jul 21 21:36:18.530279 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.530259 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.532626 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.532604 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-bacc91\"/\"kube-root-ca.crt\"" Jul 21 21:36:18.532734 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.532647 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-bacc91\"/\"openshift-service-ca.crt\"" Jul 21 21:36:18.533560 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.533540 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-bacc91\"/\"test-model-bacc91-kserve-self-signed-certs\"" Jul 21 21:36:18.533560 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.533557 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-bacc91\"/\"default-dockercfg-l5rdb\"" Jul 21 21:36:18.539792 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.539769 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw"] Jul 21 21:36:18.608023 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.607999 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e711ae-1427-403f-a57c-f64caabb67ee-tls-certs\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.608153 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.608029 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-model-cache\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.608153 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.608064 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-home\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.608153 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.608132 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-dshm\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.608254 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.608181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-tmp-dir\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.608254 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.608229 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92trk\" (UniqueName: \"kubernetes.io/projected/a0e711ae-1427-403f-a57c-f64caabb67ee-kube-api-access-92trk\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.709350 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.709310 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e711ae-1427-403f-a57c-f64caabb67ee-tls-certs\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.709350 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.709356 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-model-cache\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.709533 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.709396 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-home\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.709533 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.709422 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-dshm\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.709533 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.709441 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-tmp-dir\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.709533 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.709467 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92trk\" (UniqueName: \"kubernetes.io/projected/a0e711ae-1427-403f-a57c-f64caabb67ee-kube-api-access-92trk\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.709933 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.709898 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-home\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.710027 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.709940 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-model-cache\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.710027 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.709955 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-tmp-dir\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.711886 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.711869 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-dshm\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.712062 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.712046 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e711ae-1427-403f-a57c-f64caabb67ee-tls-certs\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.717672 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.717652 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92trk\" (UniqueName: \"kubernetes.io/projected/a0e711ae-1427-403f-a57c-f64caabb67ee-kube-api-access-92trk\") pod \"test-model-bacc91-kserve-795bc84c47-dhgnw\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.840735 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.840702 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:18.971320 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:18.971258 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw"] Jul 21 21:36:18.973668 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:36:18.973644 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0e711ae_1427_403f_a57c_f64caabb67ee.slice/crio-af01eb5770296002970d0b8fc1d97fe7861dff9beece51c547045862f6e89101 WatchSource:0}: Error finding container af01eb5770296002970d0b8fc1d97fe7861dff9beece51c547045862f6e89101: Status 404 returned error can't find the container with id af01eb5770296002970d0b8fc1d97fe7861dff9beece51c547045862f6e89101 Jul 21 21:36:19.046337 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:19.046309 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" event={"ID":"a0e711ae-1427-403f-a57c-f64caabb67ee","Type":"ContainerStarted","Data":"804747c44f06fec80bf659e48de8537e4a9f275115eec8a16c48bb67b9b1a00a"} Jul 21 21:36:19.046436 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:19.046347 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" event={"ID":"a0e711ae-1427-403f-a57c-f64caabb67ee","Type":"ContainerStarted","Data":"af01eb5770296002970d0b8fc1d97fe7861dff9beece51c547045862f6e89101"} Jul 21 21:36:19.064096 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:19.064041 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" podStartSLOduration=1.064019286 podStartE2EDuration="1.064019286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:36:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:36:19.061549067 +0000 UTC m=+2158.672302452" watchObservedRunningTime="2026-07-21 21:36:19.064019286 +0000 UTC m=+2158.674772670" Jul 21 21:36:28.841661 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:28.841621 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:28.842078 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:28.841704 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:28.854053 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:28.854025 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:29.099918 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:29.099831 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:36:34.503653 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:36:34.503612 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:37:05.043487 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.043456 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9"] Jul 21 21:37:05.047055 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.047033 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.049555 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.049532 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9e03c3\"/\"kube-root-ca.crt\"" Jul 21 21:37:05.050550 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.050531 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9e03c3\"/\"openshift-service-ca.crt\"" Jul 21 21:37:05.050550 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.050541 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9e03c3\"/\"default-dockercfg-zj7jm\"" Jul 21 21:37:05.050724 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.050541 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9e03c3\"/\"test-model-9e03c3-kserve-self-signed-certs\"" Jul 21 21:37:05.055883 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.055860 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9"] Jul 21 21:37:05.073286 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.073262 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-model-cache\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.073366 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.073304 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-dshm\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.073366 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.073328 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpct8\" (UniqueName: \"kubernetes.io/projected/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-kube-api-access-wpct8\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.073434 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.073382 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-home\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.073472 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.073450 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tmp-dir\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.073472 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.073465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tls-certs\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.173824 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.173776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-home\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.173973 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.173839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tmp-dir\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.173973 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.173946 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tls-certs\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.174054 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.174006 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-model-cache\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.174054 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.174047 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-dshm\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.174141 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.174075 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpct8\" (UniqueName: \"kubernetes.io/projected/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-kube-api-access-wpct8\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.174230 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.174210 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tmp-dir\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.174284 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.174226 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-home\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.174442 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.174421 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-model-cache\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.176242 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.176224 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-dshm\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.176535 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.176516 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tls-certs\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.185418 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.185396 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpct8\" (UniqueName: \"kubernetes.io/projected/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-kube-api-access-wpct8\") pod \"test-model-9e03c3-kserve-6464885867-gjkq9\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.356853 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.356789 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:05.486340 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:05.486315 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9"] Jul 21 21:37:05.488886 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:37:05.488860 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1f30716_5441_4da7_bbc6_8a6e84ef34f2.slice/crio-befaa0370e5becf89547d4da29f2d757a85d3fb3f674a34351210b21adfe94bb WatchSource:0}: Error finding container befaa0370e5becf89547d4da29f2d757a85d3fb3f674a34351210b21adfe94bb: Status 404 returned error can't find the container with id befaa0370e5becf89547d4da29f2d757a85d3fb3f674a34351210b21adfe94bb Jul 21 21:37:06.248614 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:06.248560 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" event={"ID":"d1f30716-5441-4da7-bbc6-8a6e84ef34f2","Type":"ContainerStarted","Data":"1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c"} Jul 21 21:37:06.248979 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:06.248621 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" event={"ID":"d1f30716-5441-4da7-bbc6-8a6e84ef34f2","Type":"ContainerStarted","Data":"befaa0370e5becf89547d4da29f2d757a85d3fb3f674a34351210b21adfe94bb"} Jul 21 21:37:06.266828 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:06.266782 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" podStartSLOduration=1.266766986 podStartE2EDuration="1.266766986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:37:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:37:06.265337467 +0000 UTC m=+2205.876090852" watchObservedRunningTime="2026-07-21 21:37:06.266766986 +0000 UTC m=+2205.877520370" Jul 21 21:37:09.502186 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:09.502151 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:37:15.357296 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:15.357255 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:15.357776 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:15.357307 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:15.369671 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:15.369647 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:16.303043 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:16.303013 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:37:22.701348 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:22.701312 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:37:45.248364 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:45.248269 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw"] Jul 21 21:37:46.218878 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:46.218833 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9"] Jul 21 21:37:50.701547 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:37:50.701507 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:38:00.269271 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.269191 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" podUID="a0e711ae-1427-403f-a57c-f64caabb67ee" containerName="main" containerID="cri-o://804747c44f06fec80bf659e48de8537e4a9f275115eec8a16c48bb67b9b1a00a" gracePeriod=45 Jul 21 21:38:00.500037 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.500002 2568 generic.go:356] "Generic (PLEG): container finished" podID="a0e711ae-1427-403f-a57c-f64caabb67ee" containerID="804747c44f06fec80bf659e48de8537e4a9f275115eec8a16c48bb67b9b1a00a" exitCode=0 Jul 21 21:38:00.500212 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.500075 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" event={"ID":"a0e711ae-1427-403f-a57c-f64caabb67ee","Type":"ContainerDied","Data":"804747c44f06fec80bf659e48de8537e4a9f275115eec8a16c48bb67b9b1a00a"} Jul 21 21:38:00.522552 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.522475 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:38:00.544849 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.544818 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92trk\" (UniqueName: \"kubernetes.io/projected/a0e711ae-1427-403f-a57c-f64caabb67ee-kube-api-access-92trk\") pod \"a0e711ae-1427-403f-a57c-f64caabb67ee\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " Jul 21 21:38:00.545037 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.544939 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-dshm\") pod \"a0e711ae-1427-403f-a57c-f64caabb67ee\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " Jul 21 21:38:00.545037 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.544973 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-model-cache\") pod \"a0e711ae-1427-403f-a57c-f64caabb67ee\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " Jul 21 21:38:00.545037 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.544999 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-tmp-dir\") pod \"a0e711ae-1427-403f-a57c-f64caabb67ee\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " Jul 21 21:38:00.545199 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.545036 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-home\") pod \"a0e711ae-1427-403f-a57c-f64caabb67ee\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " Jul 21 21:38:00.545199 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.545073 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e711ae-1427-403f-a57c-f64caabb67ee-tls-certs\") pod \"a0e711ae-1427-403f-a57c-f64caabb67ee\" (UID: \"a0e711ae-1427-403f-a57c-f64caabb67ee\") " Jul 21 21:38:00.545299 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.545223 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-model-cache" (OuterVolumeSpecName: "model-cache") pod "a0e711ae-1427-403f-a57c-f64caabb67ee" (UID: "a0e711ae-1427-403f-a57c-f64caabb67ee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:38:00.545359 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.545298 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a0e711ae-1427-403f-a57c-f64caabb67ee" (UID: "a0e711ae-1427-403f-a57c-f64caabb67ee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:38:00.545476 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.545409 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-model-cache\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:00.545476 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.545432 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-home" (OuterVolumeSpecName: "home") pod "a0e711ae-1427-403f-a57c-f64caabb67ee" (UID: "a0e711ae-1427-403f-a57c-f64caabb67ee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:38:00.545476 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.545451 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-tmp-dir\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:00.547507 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.547471 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e711ae-1427-403f-a57c-f64caabb67ee-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a0e711ae-1427-403f-a57c-f64caabb67ee" (UID: "a0e711ae-1427-403f-a57c-f64caabb67ee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:38:00.547507 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.547484 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e711ae-1427-403f-a57c-f64caabb67ee-kube-api-access-92trk" (OuterVolumeSpecName: "kube-api-access-92trk") pod "a0e711ae-1427-403f-a57c-f64caabb67ee" (UID: "a0e711ae-1427-403f-a57c-f64caabb67ee"). InnerVolumeSpecName "kube-api-access-92trk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:38:00.547676 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.547576 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-dshm" (OuterVolumeSpecName: "dshm") pod "a0e711ae-1427-403f-a57c-f64caabb67ee" (UID: "a0e711ae-1427-403f-a57c-f64caabb67ee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:38:00.646579 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.646537 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e711ae-1427-403f-a57c-f64caabb67ee-tls-certs\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:00.646579 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.646572 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92trk\" (UniqueName: \"kubernetes.io/projected/a0e711ae-1427-403f-a57c-f64caabb67ee-kube-api-access-92trk\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:00.646579 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.646620 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-dshm\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:00.646861 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:00.646635 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e711ae-1427-403f-a57c-f64caabb67ee-home\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:01.242292 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.242193 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" podUID="d1f30716-5441-4da7-bbc6-8a6e84ef34f2" containerName="main" containerID="cri-o://1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c" gracePeriod=45 Jul 21 21:38:01.495476 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.495399 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:38:01.505509 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.505471 2568 generic.go:356] "Generic (PLEG): container finished" podID="d1f30716-5441-4da7-bbc6-8a6e84ef34f2" containerID="1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c" exitCode=0 Jul 21 21:38:01.505721 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.505525 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" Jul 21 21:38:01.505721 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.505559 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" event={"ID":"d1f30716-5441-4da7-bbc6-8a6e84ef34f2","Type":"ContainerDied","Data":"1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c"} Jul 21 21:38:01.505721 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.505635 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9" event={"ID":"d1f30716-5441-4da7-bbc6-8a6e84ef34f2","Type":"ContainerDied","Data":"befaa0370e5becf89547d4da29f2d757a85d3fb3f674a34351210b21adfe94bb"} Jul 21 21:38:01.505721 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.505656 2568 scope.go:117] "RemoveContainer" containerID="1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c" Jul 21 21:38:01.507129 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.507101 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" event={"ID":"a0e711ae-1427-403f-a57c-f64caabb67ee","Type":"ContainerDied","Data":"af01eb5770296002970d0b8fc1d97fe7861dff9beece51c547045862f6e89101"} Jul 21 21:38:01.507245 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.507144 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw" Jul 21 21:38:01.516029 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.515999 2568 scope.go:117] "RemoveContainer" containerID="1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c" Jul 21 21:38:01.516410 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:38:01.516385 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c\": container with ID starting with 1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c not found: ID does not exist" containerID="1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c" Jul 21 21:38:01.516515 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.516428 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c"} err="failed to get container status \"1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c\": rpc error: code = NotFound desc = could not find container \"1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c\": container with ID starting with 1504a3b65387271511bb484ccf2a45cbee1f5d27853f03507d31fd294e31d50c not found: ID does not exist" Jul 21 21:38:01.516515 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.516455 2568 scope.go:117] "RemoveContainer" containerID="804747c44f06fec80bf659e48de8537e4a9f275115eec8a16c48bb67b9b1a00a" Jul 21 21:38:01.528715 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.528677 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw"] Jul 21 21:38:01.532668 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.532637 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-bacc91/test-model-bacc91-kserve-795bc84c47-dhgnw"] Jul 21 21:38:01.554996 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.554952 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tls-certs\") pod \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " Jul 21 21:38:01.554996 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.554998 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-dshm\") pod \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " Jul 21 21:38:01.555262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.555018 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-model-cache\") pod \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " Jul 21 21:38:01.555262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.555040 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tmp-dir\") pod \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " Jul 21 21:38:01.555262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.555074 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpct8\" (UniqueName: \"kubernetes.io/projected/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-kube-api-access-wpct8\") pod \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " Jul 21 21:38:01.555262 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.555105 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-home\") pod \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\" (UID: \"d1f30716-5441-4da7-bbc6-8a6e84ef34f2\") " Jul 21 21:38:01.555464 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.555296 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1f30716-5441-4da7-bbc6-8a6e84ef34f2" (UID: "d1f30716-5441-4da7-bbc6-8a6e84ef34f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:38:01.555464 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.555364 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d1f30716-5441-4da7-bbc6-8a6e84ef34f2" (UID: "d1f30716-5441-4da7-bbc6-8a6e84ef34f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:38:01.555572 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.555543 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-home" (OuterVolumeSpecName: "home") pod "d1f30716-5441-4da7-bbc6-8a6e84ef34f2" (UID: "d1f30716-5441-4da7-bbc6-8a6e84ef34f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:38:01.557621 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.557566 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-dshm" (OuterVolumeSpecName: "dshm") pod "d1f30716-5441-4da7-bbc6-8a6e84ef34f2" (UID: "d1f30716-5441-4da7-bbc6-8a6e84ef34f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:38:01.558004 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.557982 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-kube-api-access-wpct8" (OuterVolumeSpecName: "kube-api-access-wpct8") pod "d1f30716-5441-4da7-bbc6-8a6e84ef34f2" (UID: "d1f30716-5441-4da7-bbc6-8a6e84ef34f2"). InnerVolumeSpecName "kube-api-access-wpct8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:38:01.558111 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.558089 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1f30716-5441-4da7-bbc6-8a6e84ef34f2" (UID: "d1f30716-5441-4da7-bbc6-8a6e84ef34f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:38:01.656044 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.655998 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-home\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:01.656044 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.656033 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tls-certs\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:01.656044 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.656043 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-dshm\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:01.656044 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.656053 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-model-cache\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:01.656318 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.656061 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-tmp-dir\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:01.656318 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.656073 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpct8\" (UniqueName: \"kubernetes.io/projected/d1f30716-5441-4da7-bbc6-8a6e84ef34f2-kube-api-access-wpct8\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:38:01.829685 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.829633 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9"] Jul 21 21:38:01.835579 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:01.835540 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-9e03c3/test-model-9e03c3-kserve-6464885867-gjkq9"] Jul 21 21:38:02.912285 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:02.912248 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0e711ae-1427-403f-a57c-f64caabb67ee" path="/var/lib/kubelet/pods/a0e711ae-1427-403f-a57c-f64caabb67ee/volumes" Jul 21 21:38:02.912725 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:02.912671 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1f30716-5441-4da7-bbc6-8a6e84ef34f2" path="/var/lib/kubelet/pods/d1f30716-5441-4da7-bbc6-8a6e84ef34f2/volumes" Jul 21 21:38:05.901221 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:05.901185 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:38:21.797610 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:21.793246 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:38:40.105504 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:40.105466 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:38:57.604220 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:38:57.604182 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:39:14.899198 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:39:14.899154 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:39:42.498663 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:39:42.498623 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:39:57.499437 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:39:57.499402 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:40:13.298830 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:13.298788 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:40:21.015647 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:21.015613 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:40:21.018038 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:21.018012 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:40:21.020724 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:21.020703 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:40:21.029065 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:21.029039 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:40:21.061707 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:21.061676 2568 scope.go:117] "RemoveContainer" containerID="dd62ed7bd3502283eed55171e4ebe84bfc75188c3f92cfa5642b3f5e61103429" Jul 21 21:40:30.737310 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.737271 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf"] Jul 21 21:40:30.737806 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.737773 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1f30716-5441-4da7-bbc6-8a6e84ef34f2" containerName="main" Jul 21 21:40:30.737806 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.737790 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1f30716-5441-4da7-bbc6-8a6e84ef34f2" containerName="main" Jul 21 21:40:30.737919 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.737810 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e711ae-1427-403f-a57c-f64caabb67ee" containerName="main" Jul 21 21:40:30.737919 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.737819 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e711ae-1427-403f-a57c-f64caabb67ee" containerName="main" Jul 21 21:40:30.737919 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.737916 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1f30716-5441-4da7-bbc6-8a6e84ef34f2" containerName="main" Jul 21 21:40:30.738055 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.737928 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e711ae-1427-403f-a57c-f64caabb67ee" containerName="main" Jul 21 21:40:30.740956 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.740930 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.744538 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.744510 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e584ac\"/\"kube-root-ca.crt\"" Jul 21 21:40:30.744538 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.744524 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e584ac\"/\"default-dockercfg-jqqs8\"" Jul 21 21:40:30.744752 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.744533 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e584ac\"/\"openshift-service-ca.crt\"" Jul 21 21:40:30.744752 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.744509 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e584ac\"/\"auth-test-model-e584ac-kserve-self-signed-certs\"" Jul 21 21:40:30.750989 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.750961 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf"] Jul 21 21:40:30.758526 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.758489 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-dshm\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.758716 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.758536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-tmp-dir\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.758716 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.758632 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwwgp\" (UniqueName: \"kubernetes.io/projected/8d77e9a7-73af-404b-b139-bc8826af7c35-kube-api-access-cwwgp\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.758716 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.758672 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-model-cache\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.758716 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.758705 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d77e9a7-73af-404b-b139-bc8826af7c35-tls-certs\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.758912 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.758730 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-home\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.859873 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.859823 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-dshm\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.859873 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.859869 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-tmp-dir\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.860138 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.859908 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwwgp\" (UniqueName: \"kubernetes.io/projected/8d77e9a7-73af-404b-b139-bc8826af7c35-kube-api-access-cwwgp\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.860138 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.859931 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-model-cache\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.860138 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.859951 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d77e9a7-73af-404b-b139-bc8826af7c35-tls-certs\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.860138 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.859968 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-home\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.860379 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.860337 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-home\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.860434 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.860398 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-tmp-dir\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.860487 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.860468 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-model-cache\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.862401 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.862378 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-dshm\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.862770 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.862751 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d77e9a7-73af-404b-b139-bc8826af7c35-tls-certs\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:30.869423 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:30.869392 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwwgp\" (UniqueName: \"kubernetes.io/projected/8d77e9a7-73af-404b-b139-bc8826af7c35-kube-api-access-cwwgp\") pod \"auth-test-model-e584ac-kserve-b89cd64f9-85cpf\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:31.050514 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:31.050470 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:31.197047 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:31.197017 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf"] Jul 21 21:40:31.199866 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:40:31.199837 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d77e9a7_73af_404b_b139_bc8826af7c35.slice/crio-a919dca4fb2142e473da619b360adc1cc94bd750ebda951566ce1137965a57ef WatchSource:0}: Error finding container a919dca4fb2142e473da619b360adc1cc94bd750ebda951566ce1137965a57ef: Status 404 returned error can't find the container with id a919dca4fb2142e473da619b360adc1cc94bd750ebda951566ce1137965a57ef Jul 21 21:40:31.210975 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:31.210931 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" event={"ID":"8d77e9a7-73af-404b-b139-bc8826af7c35","Type":"ContainerStarted","Data":"a919dca4fb2142e473da619b360adc1cc94bd750ebda951566ce1137965a57ef"} Jul 21 21:40:32.217930 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:32.217889 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" event={"ID":"8d77e9a7-73af-404b-b139-bc8826af7c35","Type":"ContainerStarted","Data":"818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06"} Jul 21 21:40:32.235967 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:32.235906 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" podStartSLOduration=2.235889834 podStartE2EDuration="2.235889834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:40:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:40:32.23424594 +0000 UTC m=+2411.844999317" watchObservedRunningTime="2026-07-21 21:40:32.235889834 +0000 UTC m=+2411.846643328" Jul 21 21:40:33.792062 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:33.792014 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:40:41.050655 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:41.050613 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:41.051087 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:41.050673 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:41.063288 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:41.063261 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:41.277129 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:41.277097 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:40:56.695018 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:40:56.694977 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:41:27.692457 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:41:27.692418 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:41:41.106140 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:41:41.106102 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:41:58.191642 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:41:58.191607 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:42:15.903273 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:42:15.903238 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:42:34.594462 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:42:34.594431 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:42:49.618748 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:42:49.618714 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:43:04.331856 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:04.331824 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf"] Jul 21 21:43:08.004370 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:08.004334 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:43:15.050631 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.050580 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj"] Jul 21 21:43:15.053842 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.053825 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.056484 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.056459 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e584ac\"/\"sub-test-model-e584ac-kserve-self-signed-certs\"" Jul 21 21:43:15.066872 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.066848 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj"] Jul 21 21:43:15.070154 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.070134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-tmp-dir\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.070253 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.070167 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-model-cache\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.070253 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.070196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-home\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.070359 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.070268 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-dshm\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.070359 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.070339 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlxr5\" (UniqueName: \"kubernetes.io/projected/44592d14-9adb-41ea-9a1a-55e3d2493d25-kube-api-access-dlxr5\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.070455 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.070402 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44592d14-9adb-41ea-9a1a-55e3d2493d25-tls-certs\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.171247 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.171210 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-tmp-dir\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.171395 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.171256 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-model-cache\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.171395 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.171303 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-home\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.171395 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.171340 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-dshm\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.171395 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.171371 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlxr5\" (UniqueName: \"kubernetes.io/projected/44592d14-9adb-41ea-9a1a-55e3d2493d25-kube-api-access-dlxr5\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.171619 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.171408 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44592d14-9adb-41ea-9a1a-55e3d2493d25-tls-certs\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.171755 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.171731 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-model-cache\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.171816 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.171774 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-tmp-dir\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.171816 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.171789 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-home\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.173796 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.173779 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-dshm\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.174072 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.174054 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44592d14-9adb-41ea-9a1a-55e3d2493d25-tls-certs\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.180921 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.180899 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlxr5\" (UniqueName: \"kubernetes.io/projected/44592d14-9adb-41ea-9a1a-55e3d2493d25-kube-api-access-dlxr5\") pod \"sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.363514 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.363434 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:15.494782 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.494758 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj"] Jul 21 21:43:15.497060 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:43:15.497030 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44592d14_9adb_41ea_9a1a_55e3d2493d25.slice/crio-d911558cbbd08858f06c3780b6a16165adf6320d15bad7cf16a9c6d104c80a54 WatchSource:0}: Error finding container d911558cbbd08858f06c3780b6a16165adf6320d15bad7cf16a9c6d104c80a54: Status 404 returned error can't find the container with id d911558cbbd08858f06c3780b6a16165adf6320d15bad7cf16a9c6d104c80a54 Jul 21 21:43:15.941339 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.941307 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" event={"ID":"44592d14-9adb-41ea-9a1a-55e3d2493d25","Type":"ContainerStarted","Data":"f0259a3aff813307c2ea60b194fe82d23da17e385b17a58990682009bb5bf46a"} Jul 21 21:43:15.941339 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.941342 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" event={"ID":"44592d14-9adb-41ea-9a1a-55e3d2493d25","Type":"ContainerStarted","Data":"d911558cbbd08858f06c3780b6a16165adf6320d15bad7cf16a9c6d104c80a54"} Jul 21 21:43:15.964948 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:15.964891 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" podStartSLOduration=0.964850244 podStartE2EDuration="964.850244ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:43:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:43:15.963916031 +0000 UTC m=+2575.574669418" watchObservedRunningTime="2026-07-21 21:43:15.964850244 +0000 UTC m=+2575.575603630" Jul 21 21:43:19.352366 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.352311 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" podUID="8d77e9a7-73af-404b-b139-bc8826af7c35" containerName="main" containerID="cri-o://818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06" gracePeriod=45 Jul 21 21:43:19.601925 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.601899 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:43:19.708376 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.708293 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-dshm\") pod \"8d77e9a7-73af-404b-b139-bc8826af7c35\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " Jul 21 21:43:19.708376 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.708354 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d77e9a7-73af-404b-b139-bc8826af7c35-tls-certs\") pod \"8d77e9a7-73af-404b-b139-bc8826af7c35\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " Jul 21 21:43:19.708631 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.708395 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-home\") pod \"8d77e9a7-73af-404b-b139-bc8826af7c35\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " Jul 21 21:43:19.708631 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.708425 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwwgp\" (UniqueName: \"kubernetes.io/projected/8d77e9a7-73af-404b-b139-bc8826af7c35-kube-api-access-cwwgp\") pod \"8d77e9a7-73af-404b-b139-bc8826af7c35\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " Jul 21 21:43:19.708631 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.708446 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-tmp-dir\") pod \"8d77e9a7-73af-404b-b139-bc8826af7c35\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " Jul 21 21:43:19.708631 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.708470 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-model-cache\") pod \"8d77e9a7-73af-404b-b139-bc8826af7c35\" (UID: \"8d77e9a7-73af-404b-b139-bc8826af7c35\") " Jul 21 21:43:19.708828 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.708732 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-home" (OuterVolumeSpecName: "home") pod "8d77e9a7-73af-404b-b139-bc8826af7c35" (UID: "8d77e9a7-73af-404b-b139-bc8826af7c35"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:43:19.708828 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.708801 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8d77e9a7-73af-404b-b139-bc8826af7c35" (UID: "8d77e9a7-73af-404b-b139-bc8826af7c35"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:43:19.708896 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.708823 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-model-cache" (OuterVolumeSpecName: "model-cache") pod "8d77e9a7-73af-404b-b139-bc8826af7c35" (UID: "8d77e9a7-73af-404b-b139-bc8826af7c35"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:43:19.710815 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.710781 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-dshm" (OuterVolumeSpecName: "dshm") pod "8d77e9a7-73af-404b-b139-bc8826af7c35" (UID: "8d77e9a7-73af-404b-b139-bc8826af7c35"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:43:19.710941 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.710829 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d77e9a7-73af-404b-b139-bc8826af7c35-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8d77e9a7-73af-404b-b139-bc8826af7c35" (UID: "8d77e9a7-73af-404b-b139-bc8826af7c35"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:43:19.710941 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.710850 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d77e9a7-73af-404b-b139-bc8826af7c35-kube-api-access-cwwgp" (OuterVolumeSpecName: "kube-api-access-cwwgp") pod "8d77e9a7-73af-404b-b139-bc8826af7c35" (UID: "8d77e9a7-73af-404b-b139-bc8826af7c35"). InnerVolumeSpecName "kube-api-access-cwwgp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:43:19.809122 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.809085 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-home\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:43:19.809122 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.809119 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwwgp\" (UniqueName: \"kubernetes.io/projected/8d77e9a7-73af-404b-b139-bc8826af7c35-kube-api-access-cwwgp\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:43:19.809122 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.809129 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-tmp-dir\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:43:19.809379 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.809139 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-model-cache\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:43:19.809379 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.809147 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d77e9a7-73af-404b-b139-bc8826af7c35-dshm\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:43:19.809379 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.809154 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d77e9a7-73af-404b-b139-bc8826af7c35-tls-certs\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:43:19.959357 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.959279 2568 generic.go:356] "Generic (PLEG): container finished" podID="8d77e9a7-73af-404b-b139-bc8826af7c35" containerID="818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06" exitCode=0 Jul 21 21:43:19.959357 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.959329 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" Jul 21 21:43:19.959357 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.959340 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" event={"ID":"8d77e9a7-73af-404b-b139-bc8826af7c35","Type":"ContainerDied","Data":"818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06"} Jul 21 21:43:19.959612 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.959372 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf" event={"ID":"8d77e9a7-73af-404b-b139-bc8826af7c35","Type":"ContainerDied","Data":"a919dca4fb2142e473da619b360adc1cc94bd750ebda951566ce1137965a57ef"} Jul 21 21:43:19.959612 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.959389 2568 scope.go:117] "RemoveContainer" containerID="818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06" Jul 21 21:43:19.968355 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.968336 2568 scope.go:117] "RemoveContainer" containerID="818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06" Jul 21 21:43:19.968624 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:43:19.968604 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06\": container with ID starting with 818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06 not found: ID does not exist" containerID="818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06" Jul 21 21:43:19.968681 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.968635 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06"} err="failed to get container status \"818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06\": rpc error: code = NotFound desc = could not find container \"818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06\": container with ID starting with 818a8d6a2bd2f66cf39d4dd5e153c765b0494d3119d595c9ef2129fe4a5a1f06 not found: ID does not exist" Jul 21 21:43:19.981835 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.981815 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf"] Jul 21 21:43:19.987235 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:19.987213 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/auth-test-model-e584ac-kserve-b89cd64f9-85cpf"] Jul 21 21:43:20.912778 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:20.912745 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d77e9a7-73af-404b-b139-bc8826af7c35" path="/var/lib/kubelet/pods/8d77e9a7-73af-404b-b139-bc8826af7c35/volumes" Jul 21 21:43:21.207991 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:21.207910 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:43:25.364655 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:25.364625 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:25.365124 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:25.364671 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:25.376709 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:25.376683 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:26.003982 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:26.003954 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:43:35.499783 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:35.499750 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:43:51.108124 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:43:51.108045 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:44:09.092805 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:09.092767 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj"] Jul 21 21:44:10.596942 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:10.596911 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:44:19.261619 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.260610 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr"] Jul 21 21:44:19.261619 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.261334 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d77e9a7-73af-404b-b139-bc8826af7c35" containerName="main" Jul 21 21:44:19.261619 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.261353 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d77e9a7-73af-404b-b139-bc8826af7c35" containerName="main" Jul 21 21:44:19.261619 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.261536 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d77e9a7-73af-404b-b139-bc8826af7c35" containerName="main" Jul 21 21:44:19.265938 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.265911 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.269523 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.269501 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e584ac\"/\"rate-test-model-e584ac-kserve-self-signed-certs\"" Jul 21 21:44:19.284840 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.284818 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr"] Jul 21 21:44:19.295581 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.295558 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tls-certs\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.295700 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.295610 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m66r5\" (UniqueName: \"kubernetes.io/projected/4d4f510c-f5b9-463d-ae98-b7ee3e899460-kube-api-access-m66r5\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.295763 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.295688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-model-cache\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.295817 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.295777 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-dshm\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.295878 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.295829 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-home\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.295878 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.295855 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tmp-dir\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.397103 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.397065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-home\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.397103 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.397100 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tmp-dir\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.397640 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.397124 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tls-certs\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.397640 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.397140 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m66r5\" (UniqueName: \"kubernetes.io/projected/4d4f510c-f5b9-463d-ae98-b7ee3e899460-kube-api-access-m66r5\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.397640 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.397180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-model-cache\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.397640 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.397212 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-dshm\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.397640 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.397485 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tmp-dir\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.397640 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.397567 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-home\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.397640 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.397628 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-model-cache\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.399616 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.399572 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-dshm\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.399778 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.399759 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tls-certs\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.405535 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.405512 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m66r5\" (UniqueName: \"kubernetes.io/projected/4d4f510c-f5b9-463d-ae98-b7ee3e899460-kube-api-access-m66r5\") pod \"rate-test-model-e584ac-kserve-7669dfd99f-scjwr\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.575405 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.575363 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:19.703864 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:19.703828 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr"] Jul 21 21:44:19.706354 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:44:19.706325 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d4f510c_f5b9_463d_ae98_b7ee3e899460.slice/crio-2520f051b653551cfd08eed7dfa335da3eeca8bd8456f49a89f49ff8ce632cf1 WatchSource:0}: Error finding container 2520f051b653551cfd08eed7dfa335da3eeca8bd8456f49a89f49ff8ce632cf1: Status 404 returned error can't find the container with id 2520f051b653551cfd08eed7dfa335da3eeca8bd8456f49a89f49ff8ce632cf1 Jul 21 21:44:20.231101 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:20.231062 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" event={"ID":"4d4f510c-f5b9-463d-ae98-b7ee3e899460","Type":"ContainerStarted","Data":"b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e"} Jul 21 21:44:20.231101 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:20.231105 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" event={"ID":"4d4f510c-f5b9-463d-ae98-b7ee3e899460","Type":"ContainerStarted","Data":"2520f051b653551cfd08eed7dfa335da3eeca8bd8456f49a89f49ff8ce632cf1"} Jul 21 21:44:20.249041 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:20.248990 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" podStartSLOduration=1.248976424 podStartE2EDuration="1.248976424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:44:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:44:20.247847727 +0000 UTC m=+2639.858601127" watchObservedRunningTime="2026-07-21 21:44:20.248976424 +0000 UTC m=+2639.859729808" Jul 21 21:44:24.112924 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.112876 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" podUID="44592d14-9adb-41ea-9a1a-55e3d2493d25" containerName="main" containerID="cri-o://f0259a3aff813307c2ea60b194fe82d23da17e385b17a58990682009bb5bf46a" gracePeriod=45 Jul 21 21:44:24.251411 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.251376 2568 generic.go:356] "Generic (PLEG): container finished" podID="44592d14-9adb-41ea-9a1a-55e3d2493d25" containerID="f0259a3aff813307c2ea60b194fe82d23da17e385b17a58990682009bb5bf46a" exitCode=0 Jul 21 21:44:24.251551 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.251452 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" event={"ID":"44592d14-9adb-41ea-9a1a-55e3d2493d25","Type":"ContainerDied","Data":"f0259a3aff813307c2ea60b194fe82d23da17e385b17a58990682009bb5bf46a"} Jul 21 21:44:24.355570 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.355545 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:44:24.437244 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.437178 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlxr5\" (UniqueName: \"kubernetes.io/projected/44592d14-9adb-41ea-9a1a-55e3d2493d25-kube-api-access-dlxr5\") pod \"44592d14-9adb-41ea-9a1a-55e3d2493d25\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " Jul 21 21:44:24.437244 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.437213 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-dshm\") pod \"44592d14-9adb-41ea-9a1a-55e3d2493d25\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " Jul 21 21:44:24.437411 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.437251 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-model-cache\") pod \"44592d14-9adb-41ea-9a1a-55e3d2493d25\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " Jul 21 21:44:24.437411 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.437271 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44592d14-9adb-41ea-9a1a-55e3d2493d25-tls-certs\") pod \"44592d14-9adb-41ea-9a1a-55e3d2493d25\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " Jul 21 21:44:24.437411 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.437310 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-tmp-dir\") pod \"44592d14-9adb-41ea-9a1a-55e3d2493d25\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " Jul 21 21:44:24.437411 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.437325 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-home\") pod \"44592d14-9adb-41ea-9a1a-55e3d2493d25\" (UID: \"44592d14-9adb-41ea-9a1a-55e3d2493d25\") " Jul 21 21:44:24.437643 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.437502 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-model-cache" (OuterVolumeSpecName: "model-cache") pod "44592d14-9adb-41ea-9a1a-55e3d2493d25" (UID: "44592d14-9adb-41ea-9a1a-55e3d2493d25"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:44:24.437705 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.437649 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "44592d14-9adb-41ea-9a1a-55e3d2493d25" (UID: "44592d14-9adb-41ea-9a1a-55e3d2493d25"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:44:24.437705 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.437675 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-home" (OuterVolumeSpecName: "home") pod "44592d14-9adb-41ea-9a1a-55e3d2493d25" (UID: "44592d14-9adb-41ea-9a1a-55e3d2493d25"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:44:24.439473 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.439452 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-dshm" (OuterVolumeSpecName: "dshm") pod "44592d14-9adb-41ea-9a1a-55e3d2493d25" (UID: "44592d14-9adb-41ea-9a1a-55e3d2493d25"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:44:24.439541 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.439496 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44592d14-9adb-41ea-9a1a-55e3d2493d25-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "44592d14-9adb-41ea-9a1a-55e3d2493d25" (UID: "44592d14-9adb-41ea-9a1a-55e3d2493d25"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:44:24.439580 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.439562 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44592d14-9adb-41ea-9a1a-55e3d2493d25-kube-api-access-dlxr5" (OuterVolumeSpecName: "kube-api-access-dlxr5") pod "44592d14-9adb-41ea-9a1a-55e3d2493d25" (UID: "44592d14-9adb-41ea-9a1a-55e3d2493d25"). InnerVolumeSpecName "kube-api-access-dlxr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:44:24.537987 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.537963 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlxr5\" (UniqueName: \"kubernetes.io/projected/44592d14-9adb-41ea-9a1a-55e3d2493d25-kube-api-access-dlxr5\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:44:24.537987 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.537987 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-dshm\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:44:24.538165 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.537999 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-model-cache\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:44:24.538165 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.538008 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44592d14-9adb-41ea-9a1a-55e3d2493d25-tls-certs\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:44:24.538165 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.538015 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-tmp-dir\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:44:24.538165 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.538023 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44592d14-9adb-41ea-9a1a-55e3d2493d25-home\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:44:24.805644 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:24.805609 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:44:25.257411 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:25.257319 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" Jul 21 21:44:25.257411 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:25.257315 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj" event={"ID":"44592d14-9adb-41ea-9a1a-55e3d2493d25","Type":"ContainerDied","Data":"d911558cbbd08858f06c3780b6a16165adf6320d15bad7cf16a9c6d104c80a54"} Jul 21 21:44:25.257948 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:25.257452 2568 scope.go:117] "RemoveContainer" containerID="f0259a3aff813307c2ea60b194fe82d23da17e385b17a58990682009bb5bf46a" Jul 21 21:44:25.274139 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:25.274118 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj"] Jul 21 21:44:25.278185 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:25.278165 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/sub-test-model-e584ac-kserve-85cdb5bd8c-wxshj"] Jul 21 21:44:26.911775 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:26.911736 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44592d14-9adb-41ea-9a1a-55e3d2493d25" path="/var/lib/kubelet/pods/44592d14-9adb-41ea-9a1a-55e3d2493d25/volumes" Jul 21 21:44:29.576426 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:29.576391 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:29.576928 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:29.576478 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:29.589011 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:29.588983 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:30.290790 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:30.290757 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:44:46.302325 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:44:46.302290 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:45:06.792184 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:06.792137 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:45:21.050712 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:21.050682 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:45:21.055016 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:21.054997 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:45:21.055653 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:21.055632 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:45:21.059659 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:21.059641 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77bjb_4ef7d53f-cca5-4dc9-9c36-d064660f3f99/ovn-acl-logging/0.log" Jul 21 21:45:21.995270 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:21.995238 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:45:29.509990 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:29.509958 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr"] Jul 21 21:45:31.394350 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.394319 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7427-140701826031024"] Jul 21 21:45:31.394725 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.394663 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44592d14-9adb-41ea-9a1a-55e3d2493d25" containerName="main" Jul 21 21:45:31.394725 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.394677 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="44592d14-9adb-41ea-9a1a-55e3d2493d25" containerName="main" Jul 21 21:45:31.394800 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.394730 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="44592d14-9adb-41ea-9a1a-55e3d2493d25" containerName="main" Jul 21 21:45:31.397506 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.397488 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701826031024" Jul 21 21:45:31.402874 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.402855 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cbvbk\"" Jul 21 21:45:31.415612 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.415570 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701826031024"] Jul 21 21:45:31.484306 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.484280 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b9h8\" (UniqueName: \"kubernetes.io/projected/498d3a14-763b-4ec6-990e-3485529223ae-kube-api-access-6b9h8\") pod \"test-curl-7427-140701826031024\" (UID: \"498d3a14-763b-4ec6-990e-3485529223ae\") " pod="opendatahub/test-curl-7427-140701826031024" Jul 21 21:45:31.585517 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.585491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b9h8\" (UniqueName: \"kubernetes.io/projected/498d3a14-763b-4ec6-990e-3485529223ae-kube-api-access-6b9h8\") pod \"test-curl-7427-140701826031024\" (UID: \"498d3a14-763b-4ec6-990e-3485529223ae\") " pod="opendatahub/test-curl-7427-140701826031024" Jul 21 21:45:31.593550 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.593527 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b9h8\" (UniqueName: \"kubernetes.io/projected/498d3a14-763b-4ec6-990e-3485529223ae-kube-api-access-6b9h8\") pod \"test-curl-7427-140701826031024\" (UID: \"498d3a14-763b-4ec6-990e-3485529223ae\") " pod="opendatahub/test-curl-7427-140701826031024" Jul 21 21:45:31.706839 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.706770 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701826031024" Jul 21 21:45:31.833252 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.833225 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701826031024"] Jul 21 21:45:31.835951 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:45:31.835916 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod498d3a14_763b_4ec6_990e_3485529223ae.slice/crio-d75c82b43d260507df0c4518ee532496b8c929b6f26405a95f31e2fb19690243 WatchSource:0}: Error finding container d75c82b43d260507df0c4518ee532496b8c929b6f26405a95f31e2fb19690243: Status 404 returned error can't find the container with id d75c82b43d260507df0c4518ee532496b8c929b6f26405a95f31e2fb19690243 Jul 21 21:45:31.837169 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:31.837144 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 21:45:32.550710 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:32.550682 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701826031024" event={"ID":"498d3a14-763b-4ec6-990e-3485529223ae","Type":"ContainerStarted","Data":"d75c82b43d260507df0c4518ee532496b8c929b6f26405a95f31e2fb19690243"} Jul 21 21:45:33.557051 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:33.557015 2568 generic.go:356] "Generic (PLEG): container finished" podID="498d3a14-763b-4ec6-990e-3485529223ae" containerID="d4db97ff0fc327458e3d0c1362de4dd50779f58cc8dd31abad353e154f384065" exitCode=0 Jul 21 21:45:33.557444 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:33.557104 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701826031024" event={"ID":"498d3a14-763b-4ec6-990e-3485529223ae","Type":"ContainerDied","Data":"d4db97ff0fc327458e3d0c1362de4dd50779f58cc8dd31abad353e154f384065"} Jul 21 21:45:34.688691 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.688668 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701826031024" Jul 21 21:45:34.729410 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.729386 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7427-140701826031024_498d3a14-763b-4ec6-990e-3485529223ae/test-curl-7427-140701826031024/0.log" Jul 21 21:45:34.754029 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.753997 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7427-140701826031024"] Jul 21 21:45:34.759636 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.759615 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7427-140701826031024"] Jul 21 21:45:34.812556 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.812529 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b9h8\" (UniqueName: \"kubernetes.io/projected/498d3a14-763b-4ec6-990e-3485529223ae-kube-api-access-6b9h8\") pod \"498d3a14-763b-4ec6-990e-3485529223ae\" (UID: \"498d3a14-763b-4ec6-990e-3485529223ae\") " Jul 21 21:45:34.814851 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.814800 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/498d3a14-763b-4ec6-990e-3485529223ae-kube-api-access-6b9h8" (OuterVolumeSpecName: "kube-api-access-6b9h8") pod "498d3a14-763b-4ec6-990e-3485529223ae" (UID: "498d3a14-763b-4ec6-990e-3485529223ae"). InnerVolumeSpecName "kube-api-access-6b9h8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:45:34.891817 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.891780 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7427-140701825934000"] Jul 21 21:45:34.892097 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.892085 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="498d3a14-763b-4ec6-990e-3485529223ae" containerName="test-curl-7427-140701826031024" Jul 21 21:45:34.892138 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.892100 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="498d3a14-763b-4ec6-990e-3485529223ae" containerName="test-curl-7427-140701826031024" Jul 21 21:45:34.892177 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.892167 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="498d3a14-763b-4ec6-990e-3485529223ae" containerName="test-curl-7427-140701826031024" Jul 21 21:45:34.895198 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.895183 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825934000" Jul 21 21:45:34.900928 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.900904 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701825934000"] Jul 21 21:45:34.912239 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.912213 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="498d3a14-763b-4ec6-990e-3485529223ae" path="/var/lib/kubelet/pods/498d3a14-763b-4ec6-990e-3485529223ae/volumes" Jul 21 21:45:34.913994 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:34.913972 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b9h8\" (UniqueName: \"kubernetes.io/projected/498d3a14-763b-4ec6-990e-3485529223ae-kube-api-access-6b9h8\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:35.015122 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:35.015084 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfm4b\" (UniqueName: \"kubernetes.io/projected/138fd235-2291-40d0-ade7-31c48680eae0-kube-api-access-rfm4b\") pod \"test-curl-7427-140701825934000\" (UID: \"138fd235-2291-40d0-ade7-31c48680eae0\") " pod="opendatahub/test-curl-7427-140701825934000" Jul 21 21:45:35.116438 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:35.116365 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfm4b\" (UniqueName: \"kubernetes.io/projected/138fd235-2291-40d0-ade7-31c48680eae0-kube-api-access-rfm4b\") pod \"test-curl-7427-140701825934000\" (UID: \"138fd235-2291-40d0-ade7-31c48680eae0\") " pod="opendatahub/test-curl-7427-140701825934000" Jul 21 21:45:35.125096 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:35.125062 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfm4b\" (UniqueName: \"kubernetes.io/projected/138fd235-2291-40d0-ade7-31c48680eae0-kube-api-access-rfm4b\") pod \"test-curl-7427-140701825934000\" (UID: \"138fd235-2291-40d0-ade7-31c48680eae0\") " pod="opendatahub/test-curl-7427-140701825934000" Jul 21 21:45:35.205806 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:35.205765 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825934000" Jul 21 21:45:35.330433 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:35.330411 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701825934000"] Jul 21 21:45:35.332338 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:45:35.332315 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod138fd235_2291_40d0_ade7_31c48680eae0.slice/crio-2cb3b102dd2b0fb9acd269c1481e673ff7f036843372e1f7bba1a0efde5ed318 WatchSource:0}: Error finding container 2cb3b102dd2b0fb9acd269c1481e673ff7f036843372e1f7bba1a0efde5ed318: Status 404 returned error can't find the container with id 2cb3b102dd2b0fb9acd269c1481e673ff7f036843372e1f7bba1a0efde5ed318 Jul 21 21:45:35.567650 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:35.567619 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701825934000" event={"ID":"138fd235-2291-40d0-ade7-31c48680eae0","Type":"ContainerStarted","Data":"2cb3b102dd2b0fb9acd269c1481e673ff7f036843372e1f7bba1a0efde5ed318"} Jul 21 21:45:35.568770 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:35.568754 2568 scope.go:117] "RemoveContainer" containerID="d4db97ff0fc327458e3d0c1362de4dd50779f58cc8dd31abad353e154f384065" Jul 21 21:45:35.568770 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:35.568767 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701826031024" Jul 21 21:45:36.574639 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:36.574575 2568 generic.go:356] "Generic (PLEG): container finished" podID="138fd235-2291-40d0-ade7-31c48680eae0" containerID="1cb5759b5a65e430773e85e3eaf30b42bfd83d6a342ce3345fe8d9b5dca02963" exitCode=0 Jul 21 21:45:36.575070 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:36.574659 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701825934000" event={"ID":"138fd235-2291-40d0-ade7-31c48680eae0","Type":"ContainerDied","Data":"1cb5759b5a65e430773e85e3eaf30b42bfd83d6a342ce3345fe8d9b5dca02963"} Jul 21 21:45:37.706081 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.706059 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825934000" Jul 21 21:45:37.728170 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.728148 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7427-140701825934000_138fd235-2291-40d0-ade7-31c48680eae0/test-curl-7427-140701825934000/0.log" Jul 21 21:45:37.753640 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.753612 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7427-140701825934000"] Jul 21 21:45:37.757607 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.757574 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7427-140701825934000"] Jul 21 21:45:37.838458 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.838398 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfm4b\" (UniqueName: \"kubernetes.io/projected/138fd235-2291-40d0-ade7-31c48680eae0-kube-api-access-rfm4b\") pod \"138fd235-2291-40d0-ade7-31c48680eae0\" (UID: \"138fd235-2291-40d0-ade7-31c48680eae0\") " Jul 21 21:45:37.840546 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.840520 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/138fd235-2291-40d0-ade7-31c48680eae0-kube-api-access-rfm4b" (OuterVolumeSpecName: "kube-api-access-rfm4b") pod "138fd235-2291-40d0-ade7-31c48680eae0" (UID: "138fd235-2291-40d0-ade7-31c48680eae0"). InnerVolumeSpecName "kube-api-access-rfm4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:45:37.927614 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.927572 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7427-140701825893408"] Jul 21 21:45:37.927895 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.927883 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="138fd235-2291-40d0-ade7-31c48680eae0" containerName="test-curl-7427-140701825934000" Jul 21 21:45:37.927939 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.927898 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="138fd235-2291-40d0-ade7-31c48680eae0" containerName="test-curl-7427-140701825934000" Jul 21 21:45:37.927987 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.927978 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="138fd235-2291-40d0-ade7-31c48680eae0" containerName="test-curl-7427-140701825934000" Jul 21 21:45:37.930799 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.930782 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:37.935691 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.935669 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701825893408"] Jul 21 21:45:37.939064 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:37.939037 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfm4b\" (UniqueName: \"kubernetes.io/projected/138fd235-2291-40d0-ade7-31c48680eae0-kube-api-access-rfm4b\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:38.040269 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:38.040237 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c86fs\" (UniqueName: \"kubernetes.io/projected/1fc16387-a875-4afb-84fa-ab3005d39c4a-kube-api-access-c86fs\") pod \"test-curl-7427-140701825893408\" (UID: \"1fc16387-a875-4afb-84fa-ab3005d39c4a\") " pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:38.141626 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:38.141539 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c86fs\" (UniqueName: \"kubernetes.io/projected/1fc16387-a875-4afb-84fa-ab3005d39c4a-kube-api-access-c86fs\") pod \"test-curl-7427-140701825893408\" (UID: \"1fc16387-a875-4afb-84fa-ab3005d39c4a\") " pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:38.150284 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:38.150254 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c86fs\" (UniqueName: \"kubernetes.io/projected/1fc16387-a875-4afb-84fa-ab3005d39c4a-kube-api-access-c86fs\") pod \"test-curl-7427-140701825893408\" (UID: \"1fc16387-a875-4afb-84fa-ab3005d39c4a\") " pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:38.242037 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:38.242012 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:38.369052 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:38.369028 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701825893408"] Jul 21 21:45:38.370798 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:45:38.370772 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fc16387_a875_4afb_84fa_ab3005d39c4a.slice/crio-4a76fd952c5c3d9d4abffffc30d0cc32f7ad329df119adf29b6f45edc1e2365a WatchSource:0}: Error finding container 4a76fd952c5c3d9d4abffffc30d0cc32f7ad329df119adf29b6f45edc1e2365a: Status 404 returned error can't find the container with id 4a76fd952c5c3d9d4abffffc30d0cc32f7ad329df119adf29b6f45edc1e2365a Jul 21 21:45:38.585125 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:38.585087 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701825893408" event={"ID":"1fc16387-a875-4afb-84fa-ab3005d39c4a","Type":"ContainerStarted","Data":"4a76fd952c5c3d9d4abffffc30d0cc32f7ad329df119adf29b6f45edc1e2365a"} Jul 21 21:45:38.586321 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:38.586301 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2cb3b102dd2b0fb9acd269c1481e673ff7f036843372e1f7bba1a0efde5ed318" Jul 21 21:45:38.586453 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:38.586334 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825934000" Jul 21 21:45:38.596631 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:45:38.596573 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7427-140701825934000\" is forbidden: User \"system:node:ip-10-0-130-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-26.ec2.internal' and this object" podUID="138fd235-2291-40d0-ade7-31c48680eae0" pod="opendatahub/test-curl-7427-140701825934000" Jul 21 21:45:38.911655 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:38.911624 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="138fd235-2291-40d0-ade7-31c48680eae0" path="/var/lib/kubelet/pods/138fd235-2291-40d0-ade7-31c48680eae0/volumes" Jul 21 21:45:39.592199 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:39.592170 2568 generic.go:356] "Generic (PLEG): container finished" podID="1fc16387-a875-4afb-84fa-ab3005d39c4a" containerID="15a0cc351f3c4d6650df7bdec3a5fb19d136baeb3d5162d195160b53881c2f45" exitCode=0 Jul 21 21:45:39.592398 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:39.592268 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701825893408" event={"ID":"1fc16387-a875-4afb-84fa-ab3005d39c4a","Type":"ContainerDied","Data":"15a0cc351f3c4d6650df7bdec3a5fb19d136baeb3d5162d195160b53881c2f45"} Jul 21 21:45:40.722543 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.722522 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:40.742021 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.741988 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7427-140701825893408_1fc16387-a875-4afb-84fa-ab3005d39c4a/test-curl-7427-140701825893408/0.log" Jul 21 21:45:40.764301 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.764272 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7427-140701825893408"] Jul 21 21:45:40.768558 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.768537 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7427-140701825893408"] Jul 21 21:45:40.864383 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.864315 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c86fs\" (UniqueName: \"kubernetes.io/projected/1fc16387-a875-4afb-84fa-ab3005d39c4a-kube-api-access-c86fs\") pod \"1fc16387-a875-4afb-84fa-ab3005d39c4a\" (UID: \"1fc16387-a875-4afb-84fa-ab3005d39c4a\") " Jul 21 21:45:40.866491 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.866466 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fc16387-a875-4afb-84fa-ab3005d39c4a-kube-api-access-c86fs" (OuterVolumeSpecName: "kube-api-access-c86fs") pod "1fc16387-a875-4afb-84fa-ab3005d39c4a" (UID: "1fc16387-a875-4afb-84fa-ab3005d39c4a"). InnerVolumeSpecName "kube-api-access-c86fs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:45:40.911235 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.911213 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fc16387-a875-4afb-84fa-ab3005d39c4a" path="/var/lib/kubelet/pods/1fc16387-a875-4afb-84fa-ab3005d39c4a/volumes" Jul 21 21:45:40.918056 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.918033 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7427-140701825893408"] Jul 21 21:45:40.918420 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.918404 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc16387-a875-4afb-84fa-ab3005d39c4a" containerName="test-curl-7427-140701825893408" Jul 21 21:45:40.918420 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.918422 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc16387-a875-4afb-84fa-ab3005d39c4a" containerName="test-curl-7427-140701825893408" Jul 21 21:45:40.918522 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.918480 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fc16387-a875-4afb-84fa-ab3005d39c4a" containerName="test-curl-7427-140701825893408" Jul 21 21:45:40.921430 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.921414 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:40.928220 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.928198 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701825893408"] Jul 21 21:45:40.964899 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:40.964874 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c86fs\" (UniqueName: \"kubernetes.io/projected/1fc16387-a875-4afb-84fa-ab3005d39c4a-kube-api-access-c86fs\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:41.065719 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:41.065684 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggjjz\" (UniqueName: \"kubernetes.io/projected/40c5e3d0-0a7f-4f03-9486-8c97378489bd-kube-api-access-ggjjz\") pod \"test-curl-7427-140701825893408\" (UID: \"40c5e3d0-0a7f-4f03-9486-8c97378489bd\") " pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:41.167307 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:41.167213 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjjz\" (UniqueName: \"kubernetes.io/projected/40c5e3d0-0a7f-4f03-9486-8c97378489bd-kube-api-access-ggjjz\") pod \"test-curl-7427-140701825893408\" (UID: \"40c5e3d0-0a7f-4f03-9486-8c97378489bd\") " pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:41.177478 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:41.177450 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggjjz\" (UniqueName: \"kubernetes.io/projected/40c5e3d0-0a7f-4f03-9486-8c97378489bd-kube-api-access-ggjjz\") pod \"test-curl-7427-140701825893408\" (UID: \"40c5e3d0-0a7f-4f03-9486-8c97378489bd\") " pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:41.230556 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:41.230527 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:41.357283 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:41.357232 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701825893408"] Jul 21 21:45:41.359427 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:45:41.359400 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40c5e3d0_0a7f_4f03_9486_8c97378489bd.slice/crio-5c9b9cd44acdef5e222970ac961ce6999fc5ed154e0f2e29b3e1bbc2efe0a251 WatchSource:0}: Error finding container 5c9b9cd44acdef5e222970ac961ce6999fc5ed154e0f2e29b3e1bbc2efe0a251: Status 404 returned error can't find the container with id 5c9b9cd44acdef5e222970ac961ce6999fc5ed154e0f2e29b3e1bbc2efe0a251 Jul 21 21:45:41.602377 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:41.602351 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:41.602538 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:41.602376 2568 scope.go:117] "RemoveContainer" containerID="15a0cc351f3c4d6650df7bdec3a5fb19d136baeb3d5162d195160b53881c2f45" Jul 21 21:45:41.603646 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:41.603623 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701825893408" event={"ID":"40c5e3d0-0a7f-4f03-9486-8c97378489bd","Type":"ContainerStarted","Data":"5c9b9cd44acdef5e222970ac961ce6999fc5ed154e0f2e29b3e1bbc2efe0a251"} Jul 21 21:45:42.200554 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:42.200513 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:45:42.611465 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:42.611433 2568 generic.go:356] "Generic (PLEG): container finished" podID="40c5e3d0-0a7f-4f03-9486-8c97378489bd" containerID="aeee13bb43104267fcb30a3a4626297ba127219045f1d7ce8f7fcb21dd23ad5d" exitCode=0 Jul 21 21:45:42.611656 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:42.611489 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701825893408" event={"ID":"40c5e3d0-0a7f-4f03-9486-8c97378489bd","Type":"ContainerDied","Data":"aeee13bb43104267fcb30a3a4626297ba127219045f1d7ce8f7fcb21dd23ad5d"} Jul 21 21:45:43.747963 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.747942 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:43.771302 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.771279 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7427-140701825893408_40c5e3d0-0a7f-4f03-9486-8c97378489bd/test-curl-7427-140701825893408/0.log" Jul 21 21:45:43.797033 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.797009 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7427-140701825893408"] Jul 21 21:45:43.801767 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.801748 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7427-140701825893408"] Jul 21 21:45:43.888482 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.888421 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggjjz\" (UniqueName: \"kubernetes.io/projected/40c5e3d0-0a7f-4f03-9486-8c97378489bd-kube-api-access-ggjjz\") pod \"40c5e3d0-0a7f-4f03-9486-8c97378489bd\" (UID: \"40c5e3d0-0a7f-4f03-9486-8c97378489bd\") " Jul 21 21:45:43.890522 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.890496 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c5e3d0-0a7f-4f03-9486-8c97378489bd-kube-api-access-ggjjz" (OuterVolumeSpecName: "kube-api-access-ggjjz") pod "40c5e3d0-0a7f-4f03-9486-8c97378489bd" (UID: "40c5e3d0-0a7f-4f03-9486-8c97378489bd"). InnerVolumeSpecName "kube-api-access-ggjjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:45:43.944165 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.944136 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7427-140701825894416"] Jul 21 21:45:43.944459 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.944447 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c5e3d0-0a7f-4f03-9486-8c97378489bd" containerName="test-curl-7427-140701825893408" Jul 21 21:45:43.944517 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.944463 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c5e3d0-0a7f-4f03-9486-8c97378489bd" containerName="test-curl-7427-140701825893408" Jul 21 21:45:43.944554 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.944533 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c5e3d0-0a7f-4f03-9486-8c97378489bd" containerName="test-curl-7427-140701825893408" Jul 21 21:45:43.947272 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.947257 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:43.961073 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.961051 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701825894416"] Jul 21 21:45:43.989855 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:43.989792 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggjjz\" (UniqueName: \"kubernetes.io/projected/40c5e3d0-0a7f-4f03-9486-8c97378489bd-kube-api-access-ggjjz\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:44.090790 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.090763 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgxh2\" (UniqueName: \"kubernetes.io/projected/9430f5a3-43d0-4ac0-bc55-7f4dea4ea259-kube-api-access-sgxh2\") pod \"test-curl-7427-140701825894416\" (UID: \"9430f5a3-43d0-4ac0-bc55-7f4dea4ea259\") " pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:44.192134 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.192053 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgxh2\" (UniqueName: \"kubernetes.io/projected/9430f5a3-43d0-4ac0-bc55-7f4dea4ea259-kube-api-access-sgxh2\") pod \"test-curl-7427-140701825894416\" (UID: \"9430f5a3-43d0-4ac0-bc55-7f4dea4ea259\") " pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:44.201234 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.201211 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgxh2\" (UniqueName: \"kubernetes.io/projected/9430f5a3-43d0-4ac0-bc55-7f4dea4ea259-kube-api-access-sgxh2\") pod \"test-curl-7427-140701825894416\" (UID: \"9430f5a3-43d0-4ac0-bc55-7f4dea4ea259\") " pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:44.256360 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.256337 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:44.397337 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.397306 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701825894416"] Jul 21 21:45:44.404751 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:45:44.404726 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9430f5a3_43d0_4ac0_bc55_7f4dea4ea259.slice/crio-d453973ef61ceb2ee2ce36ac6aa53ec5ac0b9d34a534f6335b1e93eb245b395d WatchSource:0}: Error finding container d453973ef61ceb2ee2ce36ac6aa53ec5ac0b9d34a534f6335b1e93eb245b395d: Status 404 returned error can't find the container with id d453973ef61ceb2ee2ce36ac6aa53ec5ac0b9d34a534f6335b1e93eb245b395d Jul 21 21:45:44.529969 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.529892 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" podUID="4d4f510c-f5b9-463d-ae98-b7ee3e899460" containerName="main" containerID="cri-o://b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e" gracePeriod=45 Jul 21 21:45:44.622759 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.622733 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:44.622759 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.622761 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c9b9cd44acdef5e222970ac961ce6999fc5ed154e0f2e29b3e1bbc2efe0a251" Jul 21 21:45:44.624022 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.623993 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701825894416" event={"ID":"9430f5a3-43d0-4ac0-bc55-7f4dea4ea259","Type":"ContainerStarted","Data":"d453973ef61ceb2ee2ce36ac6aa53ec5ac0b9d34a534f6335b1e93eb245b395d"} Jul 21 21:45:44.632803 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:45:44.632781 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7427-140701825893408\" is forbidden: User \"system:node:ip-10-0-130-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-26.ec2.internal' and this object" podUID="40c5e3d0-0a7f-4f03-9486-8c97378489bd" pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:44.795434 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.795411 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:45:44.813900 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:45:44.813865 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7427-140701825893408\" is forbidden: User \"system:node:ip-10-0-130-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-26.ec2.internal' and this object" podUID="40c5e3d0-0a7f-4f03-9486-8c97378489bd" pod="opendatahub/test-curl-7427-140701825893408" Jul 21 21:45:44.898105 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.898082 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-model-cache\") pod \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " Jul 21 21:45:44.898218 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.898113 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tmp-dir\") pod \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " Jul 21 21:45:44.898218 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.898143 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m66r5\" (UniqueName: \"kubernetes.io/projected/4d4f510c-f5b9-463d-ae98-b7ee3e899460-kube-api-access-m66r5\") pod \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " Jul 21 21:45:44.898218 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.898198 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-dshm\") pod \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " Jul 21 21:45:44.898385 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.898248 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tls-certs\") pod \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " Jul 21 21:45:44.898385 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.898270 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-home\") pod \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\" (UID: \"4d4f510c-f5b9-463d-ae98-b7ee3e899460\") " Jul 21 21:45:44.898385 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.898362 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-model-cache" (OuterVolumeSpecName: "model-cache") pod "4d4f510c-f5b9-463d-ae98-b7ee3e899460" (UID: "4d4f510c-f5b9-463d-ae98-b7ee3e899460"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:45:44.898385 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.898358 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4d4f510c-f5b9-463d-ae98-b7ee3e899460" (UID: "4d4f510c-f5b9-463d-ae98-b7ee3e899460"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:45:44.898579 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.898540 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-home" (OuterVolumeSpecName: "home") pod "4d4f510c-f5b9-463d-ae98-b7ee3e899460" (UID: "4d4f510c-f5b9-463d-ae98-b7ee3e899460"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:45:44.898579 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.898569 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-model-cache\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:44.898674 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.898580 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tmp-dir\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:44.900544 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.900518 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-dshm" (OuterVolumeSpecName: "dshm") pod "4d4f510c-f5b9-463d-ae98-b7ee3e899460" (UID: "4d4f510c-f5b9-463d-ae98-b7ee3e899460"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:45:44.900544 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.900518 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4d4f510c-f5b9-463d-ae98-b7ee3e899460" (UID: "4d4f510c-f5b9-463d-ae98-b7ee3e899460"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:45:44.900689 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.900520 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d4f510c-f5b9-463d-ae98-b7ee3e899460-kube-api-access-m66r5" (OuterVolumeSpecName: "kube-api-access-m66r5") pod "4d4f510c-f5b9-463d-ae98-b7ee3e899460" (UID: "4d4f510c-f5b9-463d-ae98-b7ee3e899460"). InnerVolumeSpecName "kube-api-access-m66r5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:45:44.913296 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.913272 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40c5e3d0-0a7f-4f03-9486-8c97378489bd" path="/var/lib/kubelet/pods/40c5e3d0-0a7f-4f03-9486-8c97378489bd/volumes" Jul 21 21:45:44.999754 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.999727 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m66r5\" (UniqueName: \"kubernetes.io/projected/4d4f510c-f5b9-463d-ae98-b7ee3e899460-kube-api-access-m66r5\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:44.999860 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.999759 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-dshm\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:44.999860 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.999778 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d4f510c-f5b9-463d-ae98-b7ee3e899460-tls-certs\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:44.999860 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:44.999794 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d4f510c-f5b9-463d-ae98-b7ee3e899460-home\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:45.635931 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:45.635880 2568 generic.go:356] "Generic (PLEG): container finished" podID="9430f5a3-43d0-4ac0-bc55-7f4dea4ea259" containerID="eb26658d6323f039f99d01bca5921bca2c39b8be274ef0d9bd97e68c7b800ffa" exitCode=0 Jul 21 21:45:45.636203 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:45.635956 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701825894416" event={"ID":"9430f5a3-43d0-4ac0-bc55-7f4dea4ea259","Type":"ContainerDied","Data":"eb26658d6323f039f99d01bca5921bca2c39b8be274ef0d9bd97e68c7b800ffa"} Jul 21 21:45:45.637431 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:45.637405 2568 generic.go:356] "Generic (PLEG): container finished" podID="4d4f510c-f5b9-463d-ae98-b7ee3e899460" containerID="b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e" exitCode=0 Jul 21 21:45:45.637549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:45.637454 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" Jul 21 21:45:45.637549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:45.637487 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" event={"ID":"4d4f510c-f5b9-463d-ae98-b7ee3e899460","Type":"ContainerDied","Data":"b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e"} Jul 21 21:45:45.637549 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:45.637542 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr" event={"ID":"4d4f510c-f5b9-463d-ae98-b7ee3e899460","Type":"ContainerDied","Data":"2520f051b653551cfd08eed7dfa335da3eeca8bd8456f49a89f49ff8ce632cf1"} Jul 21 21:45:45.637724 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:45.637563 2568 scope.go:117] "RemoveContainer" containerID="b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e" Jul 21 21:45:45.647315 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:45.647121 2568 scope.go:117] "RemoveContainer" containerID="b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e" Jul 21 21:45:45.647405 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:45:45.647388 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e\": container with ID starting with b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e not found: ID does not exist" containerID="b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e" Jul 21 21:45:45.647444 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:45.647413 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e"} err="failed to get container status \"b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e\": rpc error: code = NotFound desc = could not find container \"b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e\": container with ID starting with b3938bfc124ce2184b57ab925189c27f3a0e50c4a582078bf2ef94af800cea5e not found: ID does not exist" Jul 21 21:45:45.665973 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:45.665947 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr"] Jul 21 21:45:45.670237 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:45.670219 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e584ac/rate-test-model-e584ac-kserve-7669dfd99f-scjwr"] Jul 21 21:45:46.773951 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.773927 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:46.795397 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.795375 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7427-140701825894416_9430f5a3-43d0-4ac0-bc55-7f4dea4ea259/test-curl-7427-140701825894416/0.log" Jul 21 21:45:46.819127 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.819101 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7427-140701825894416"] Jul 21 21:45:46.822775 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.822756 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7427-140701825894416"] Jul 21 21:45:46.911672 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.911581 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d4f510c-f5b9-463d-ae98-b7ee3e899460" path="/var/lib/kubelet/pods/4d4f510c-f5b9-463d-ae98-b7ee3e899460/volumes" Jul 21 21:45:46.914615 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.914573 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgxh2\" (UniqueName: \"kubernetes.io/projected/9430f5a3-43d0-4ac0-bc55-7f4dea4ea259-kube-api-access-sgxh2\") pod \"9430f5a3-43d0-4ac0-bc55-7f4dea4ea259\" (UID: \"9430f5a3-43d0-4ac0-bc55-7f4dea4ea259\") " Jul 21 21:45:46.916734 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.916711 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9430f5a3-43d0-4ac0-bc55-7f4dea4ea259-kube-api-access-sgxh2" (OuterVolumeSpecName: "kube-api-access-sgxh2") pod "9430f5a3-43d0-4ac0-bc55-7f4dea4ea259" (UID: "9430f5a3-43d0-4ac0-bc55-7f4dea4ea259"). InnerVolumeSpecName "kube-api-access-sgxh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:45:46.962756 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.962725 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7427-140701825894416"] Jul 21 21:45:46.963080 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.963066 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d4f510c-f5b9-463d-ae98-b7ee3e899460" containerName="main" Jul 21 21:45:46.963121 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.963083 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d4f510c-f5b9-463d-ae98-b7ee3e899460" containerName="main" Jul 21 21:45:46.963121 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.963103 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9430f5a3-43d0-4ac0-bc55-7f4dea4ea259" containerName="test-curl-7427-140701825894416" Jul 21 21:45:46.963121 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.963109 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9430f5a3-43d0-4ac0-bc55-7f4dea4ea259" containerName="test-curl-7427-140701825894416" Jul 21 21:45:46.963213 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.963162 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9430f5a3-43d0-4ac0-bc55-7f4dea4ea259" containerName="test-curl-7427-140701825894416" Jul 21 21:45:46.963213 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.963169 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d4f510c-f5b9-463d-ae98-b7ee3e899460" containerName="main" Jul 21 21:45:46.966288 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.966272 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:46.973052 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:46.972948 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701825894416"] Jul 21 21:45:47.016093 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:47.016054 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgxh2\" (UniqueName: \"kubernetes.io/projected/9430f5a3-43d0-4ac0-bc55-7f4dea4ea259-kube-api-access-sgxh2\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:47.116931 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:47.116895 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f268t\" (UniqueName: \"kubernetes.io/projected/48e54ab5-9110-460d-b73b-977a10c965af-kube-api-access-f268t\") pod \"test-curl-7427-140701825894416\" (UID: \"48e54ab5-9110-460d-b73b-977a10c965af\") " pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:47.217776 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:47.217688 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f268t\" (UniqueName: \"kubernetes.io/projected/48e54ab5-9110-460d-b73b-977a10c965af-kube-api-access-f268t\") pod \"test-curl-7427-140701825894416\" (UID: \"48e54ab5-9110-460d-b73b-977a10c965af\") " pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:47.227208 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:47.227184 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f268t\" (UniqueName: \"kubernetes.io/projected/48e54ab5-9110-460d-b73b-977a10c965af-kube-api-access-f268t\") pod \"test-curl-7427-140701825894416\" (UID: \"48e54ab5-9110-460d-b73b-977a10c965af\") " pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:47.275566 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:47.275525 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:47.401311 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:47.401285 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7427-140701825894416"] Jul 21 21:45:47.403752 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:45:47.403722 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48e54ab5_9110_460d_b73b_977a10c965af.slice/crio-999bae5531d7a8e549676a00c6b5ab1a1f7490bbb2bbb5ba557b026729dc83ab WatchSource:0}: Error finding container 999bae5531d7a8e549676a00c6b5ab1a1f7490bbb2bbb5ba557b026729dc83ab: Status 404 returned error can't find the container with id 999bae5531d7a8e549676a00c6b5ab1a1f7490bbb2bbb5ba557b026729dc83ab Jul 21 21:45:47.650400 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:47.650371 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:47.650400 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:47.650380 2568 scope.go:117] "RemoveContainer" containerID="eb26658d6323f039f99d01bca5921bca2c39b8be274ef0d9bd97e68c7b800ffa" Jul 21 21:45:47.651528 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:47.651500 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701825894416" event={"ID":"48e54ab5-9110-460d-b73b-977a10c965af","Type":"ContainerStarted","Data":"999bae5531d7a8e549676a00c6b5ab1a1f7490bbb2bbb5ba557b026729dc83ab"} Jul 21 21:45:48.658910 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:48.658875 2568 generic.go:356] "Generic (PLEG): container finished" podID="48e54ab5-9110-460d-b73b-977a10c965af" containerID="7c82647cc26e5484a14153f57d3e25902eb5a700b6ec8b8c6a8983ad4ea51692" exitCode=0 Jul 21 21:45:48.659296 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:48.658952 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7427-140701825894416" event={"ID":"48e54ab5-9110-460d-b73b-977a10c965af","Type":"ContainerDied","Data":"7c82647cc26e5484a14153f57d3e25902eb5a700b6ec8b8c6a8983ad4ea51692"} Jul 21 21:45:48.912054 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:48.911975 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9430f5a3-43d0-4ac0-bc55-7f4dea4ea259" path="/var/lib/kubelet/pods/9430f5a3-43d0-4ac0-bc55-7f4dea4ea259/volumes" Jul 21 21:45:49.792986 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:49.792965 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:49.814391 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:49.814360 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7427-140701825894416_48e54ab5-9110-460d-b73b-977a10c965af/test-curl-7427-140701825894416/0.log" Jul 21 21:45:49.841455 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:49.841427 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7427-140701825894416"] Jul 21 21:45:49.847246 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:49.847223 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7427-140701825894416"] Jul 21 21:45:49.938127 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:49.938045 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f268t\" (UniqueName: \"kubernetes.io/projected/48e54ab5-9110-460d-b73b-977a10c965af-kube-api-access-f268t\") pod \"48e54ab5-9110-460d-b73b-977a10c965af\" (UID: \"48e54ab5-9110-460d-b73b-977a10c965af\") " Jul 21 21:45:49.940221 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:49.940199 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48e54ab5-9110-460d-b73b-977a10c965af-kube-api-access-f268t" (OuterVolumeSpecName: "kube-api-access-f268t") pod "48e54ab5-9110-460d-b73b-977a10c965af" (UID: "48e54ab5-9110-460d-b73b-977a10c965af"). InnerVolumeSpecName "kube-api-access-f268t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:45:50.039390 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:50.039358 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f268t\" (UniqueName: \"kubernetes.io/projected/48e54ab5-9110-460d-b73b-977a10c965af-kube-api-access-f268t\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\"" Jul 21 21:45:50.669170 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:50.669137 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:50.669170 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:50.669164 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="999bae5531d7a8e549676a00c6b5ab1a1f7490bbb2bbb5ba557b026729dc83ab" Jul 21 21:45:50.679323 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:45:50.679295 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7427-140701825894416\" is forbidden: User \"system:node:ip-10-0-130-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-26.ec2.internal' and this object" podUID="48e54ab5-9110-460d-b73b-977a10c965af" pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:50.910203 ip-10-0-130-26 kubenswrapper[2568]: E0721 21:45:50.910152 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7427-140701825894416\" is forbidden: User \"system:node:ip-10-0-130-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-26.ec2.internal' and this object" podUID="48e54ab5-9110-460d-b73b-977a10c965af" pod="opendatahub/test-curl-7427-140701825894416" Jul 21 21:45:50.911355 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:45:50.911328 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48e54ab5-9110-460d-b73b-977a10c965af" path="/var/lib/kubelet/pods/48e54ab5-9110-460d-b73b-977a10c965af/volumes" Jul 21 21:46:03.602136 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:03.602095 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:46:25.994208 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:25.994173 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8t4bw"] Jul 21 21:46:34.629358 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:34.629325 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54bdc969b5-8jjtb_85d8197e-aeb4-4cdd-8084-fa65ef54d317/authorino/0.log" Jul 21 21:46:40.939966 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:40.939934 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694dcbb678-jq7lj_fb9a9560-e187-4512-9f0b-5cdb186f1fab/copy-manifests/0.log" Jul 21 21:46:40.948008 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:40.947984 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694dcbb678-jq7lj_fb9a9560-e187-4512-9f0b-5cdb186f1fab/manager/0.log" Jul 21 21:46:43.439234 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:43.439143 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54bdc969b5-8jjtb_85d8197e-aeb4-4cdd-8084-fa65ef54d317/authorino/0.log" Jul 21 21:46:43.807756 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:43.807730 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d7k4n_b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051/kuadrant-console-plugin/0.log" Jul 21 21:46:44.210933 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:44.210846 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8t4bw_8e552cae-91ec-4f23-84ba-5a2859d47050/limitador/0.log" Jul 21 21:46:45.088367 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:45.088330 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b46444884-9ztk5_2735e6a0-9b32-480d-9261-865f9df1174a/kube-auth-proxy/0.log" Jul 21 21:46:46.499444 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:46.499416 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ndwkj_7f74a56d-3b37-43c7-9872-bd6554ba95e6/main/0.log" Jul 21 21:46:46.619347 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:46.619318 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-j8ptt_f75efee8-7d35-44f1-9ad1-e6d1d0737c32/main/0.log" Jul 21 21:46:55.957035 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:55.957006 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pchv7_681e613d-9c45-4255-be57-f765bec9fb14/global-pull-secret-syncer/0.log" Jul 21 21:46:56.079997 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:56.079964 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vxtql_a816a2e0-0ddf-42e2-ad1b-54b69582b666/konnectivity-agent/0.log" Jul 21 21:46:56.102282 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:46:56.102262 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-26.ec2.internal_916d60af6bbc560eb91de9d2f508780c/haproxy/0.log" Jul 21 21:47:00.719014 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:00.718976 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54bdc969b5-8jjtb_85d8197e-aeb4-4cdd-8084-fa65ef54d317/authorino/0.log" Jul 21 21:47:00.830010 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:00.829978 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d7k4n_b2b6cc84-ff0d-4bed-b9b9-c5bc79ea4051/kuadrant-console-plugin/0.log" Jul 21 21:47:01.187206 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:01.187178 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8t4bw_8e552cae-91ec-4f23-84ba-5a2859d47050/limitador/0.log" Jul 21 21:47:03.021678 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:03.021647 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_07189de5-d912-41d5-9b48-bdbd98d9e31a/alertmanager/0.log" Jul 21 21:47:03.056990 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:03.056959 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_07189de5-d912-41d5-9b48-bdbd98d9e31a/config-reloader/0.log" Jul 21 21:47:03.092063 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:03.092037 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_07189de5-d912-41d5-9b48-bdbd98d9e31a/kube-rbac-proxy-web/0.log" Jul 21 21:47:03.122181 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:03.122159 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_07189de5-d912-41d5-9b48-bdbd98d9e31a/kube-rbac-proxy/0.log" Jul 21 21:47:03.151151 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:03.151129 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_07189de5-d912-41d5-9b48-bdbd98d9e31a/kube-rbac-proxy-metric/0.log" Jul 21 21:47:03.189282 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:03.189252 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_07189de5-d912-41d5-9b48-bdbd98d9e31a/prom-label-proxy/0.log" Jul 21 21:47:03.214692 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:03.214671 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_07189de5-d912-41d5-9b48-bdbd98d9e31a/init-config-reloader/0.log" Jul 21 21:47:03.414741 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:03.414714 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-57dj7_f29b55a7-ff87-49d7-93d6-b1f0fc8ac44e/monitoring-plugin/0.log" Jul 21 21:47:03.626542 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:03.626515 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v6w5c_e0e5cd46-da6d-4eb3-9ff7-dd3b62961850/node-exporter/0.log" Jul 21 21:47:03.656281 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:03.656260 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v6w5c_e0e5cd46-da6d-4eb3-9ff7-dd3b62961850/kube-rbac-proxy/0.log" Jul 21 21:47:03.688105 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:03.688049 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v6w5c_e0e5cd46-da6d-4eb3-9ff7-dd3b62961850/init-textfile/0.log" Jul 21 21:47:04.091195 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:04.091168 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65c85bcf49-csjs4_c626b62c-02d3-4e14-a4a9-d54540b5ef54/telemeter-client/0.log" Jul 21 21:47:04.141047 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:04.141020 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65c85bcf49-csjs4_c626b62c-02d3-4e14-a4a9-d54540b5ef54/reload/0.log" Jul 21 21:47:04.177712 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:04.177694 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65c85bcf49-csjs4_c626b62c-02d3-4e14-a4a9-d54540b5ef54/kube-rbac-proxy/0.log" Jul 21 21:47:06.194632 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.194603 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/1.log" Jul 21 21:47:06.199389 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.199369 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ls9lw_604ec39e-5292-4177-a045-5f639fc3d042/console-operator/2.log" Jul 21 21:47:06.214015 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.213991 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk"] Jul 21 21:47:06.214296 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.214284 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48e54ab5-9110-460d-b73b-977a10c965af" containerName="test-curl-7427-140701825894416" Jul 21 21:47:06.214337 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.214298 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="48e54ab5-9110-460d-b73b-977a10c965af" containerName="test-curl-7427-140701825894416" Jul 21 21:47:06.214391 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.214382 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="48e54ab5-9110-460d-b73b-977a10c965af" containerName="test-curl-7427-140701825894416" Jul 21 21:47:06.217311 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.217294 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.219887 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.219856 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b4hjj\"/\"openshift-service-ca.crt\"" Jul 21 21:47:06.219991 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.219899 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b4hjj\"/\"kube-root-ca.crt\"" Jul 21 21:47:06.220752 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.220737 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b4hjj\"/\"default-dockercfg-p8l7g\"" Jul 21 21:47:06.228716 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.228697 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk"] Jul 21 21:47:06.265036 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.265008 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcdgq\" (UniqueName: \"kubernetes.io/projected/9043297a-aabb-4d65-aa40-db935e25520a-kube-api-access-tcdgq\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.265192 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.265042 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-proc\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.265192 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.265084 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-sys\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.265192 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.265100 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-lib-modules\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.265192 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.265122 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-podres\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.365811 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.365776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-sys\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.365963 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.365819 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-lib-modules\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.365963 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.365861 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-podres\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.365963 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.365901 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-sys\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.365963 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.365909 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcdgq\" (UniqueName: \"kubernetes.io/projected/9043297a-aabb-4d65-aa40-db935e25520a-kube-api-access-tcdgq\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.366142 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.365988 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-proc\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.366142 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.366020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-podres\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.366142 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.366031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-lib-modules\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.366142 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.366054 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9043297a-aabb-4d65-aa40-db935e25520a-proc\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.374906 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.374879 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcdgq\" (UniqueName: \"kubernetes.io/projected/9043297a-aabb-4d65-aa40-db935e25520a-kube-api-access-tcdgq\") pod \"perf-node-gather-daemonset-m96tk\" (UID: \"9043297a-aabb-4d65-aa40-db935e25520a\") " pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.527673 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.527577 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:06.662230 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:06.662204 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk"] Jul 21 21:47:06.664965 ip-10-0-130-26 kubenswrapper[2568]: W0721 21:47:06.664938 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9043297a_aabb_4d65_aa40_db935e25520a.slice/crio-37ac2c4be4253c517977016bc24d1fffbdc10979261529692662e54f78c85748 WatchSource:0}: Error finding container 37ac2c4be4253c517977016bc24d1fffbdc10979261529692662e54f78c85748: Status 404 returned error can't find the container with id 37ac2c4be4253c517977016bc24d1fffbdc10979261529692662e54f78c85748 Jul 21 21:47:07.010457 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:07.010420 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" event={"ID":"9043297a-aabb-4d65-aa40-db935e25520a","Type":"ContainerStarted","Data":"dbbcd833507286c7ce1695abceb416fdf319ad04d96d2f61ab43f9212ae2ee6a"} Jul 21 21:47:07.010654 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:07.010485 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" event={"ID":"9043297a-aabb-4d65-aa40-db935e25520a","Type":"ContainerStarted","Data":"37ac2c4be4253c517977016bc24d1fffbdc10979261529692662e54f78c85748"} Jul 21 21:47:07.010654 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:07.010620 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:07.037951 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:07.037843 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" podStartSLOduration=1.037824244 podStartE2EDuration="1.037824244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:47:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:47:07.036065201 +0000 UTC m=+2806.646818585" watchObservedRunningTime="2026-07-21 21:47:07.037824244 +0000 UTC m=+2806.648577628" Jul 21 21:47:08.470965 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:08.470936 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qflql_ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b/dns/0.log" Jul 21 21:47:08.495784 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:08.495760 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qflql_ad8bbb4c-8119-4c4d-8e7c-e4e5a5b86e4b/kube-rbac-proxy/0.log" Jul 21 21:47:08.577989 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:08.577959 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cghfw_7f05adfe-0e27-4ea2-8e61-4e03c16fba01/dns-node-resolver/0.log" Jul 21 21:47:09.177364 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:09.177328 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pmwz_65f51fb4-b664-45da-81ab-4a0d20506ad2/node-ca/0.log" Jul 21 21:47:10.454272 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:10.454243 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b46444884-9ztk5_2735e6a0-9b32-480d-9261-865f9df1174a/kube-auth-proxy/0.log" Jul 21 21:47:11.326060 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:11.326031 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7xpvd_b7561c51-58dc-4d33-89d8-974e4cc6601a/serve-healthcheck-canary/0.log" Jul 21 21:47:11.849922 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:11.849894 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n2tht_10107a18-038c-41a9-847d-c96604afd79f/kube-rbac-proxy/0.log" Jul 21 21:47:11.871431 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:11.871409 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n2tht_10107a18-038c-41a9-847d-c96604afd79f/exporter/0.log" Jul 21 21:47:11.894536 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:11.894514 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n2tht_10107a18-038c-41a9-847d-c96604afd79f/extractor/0.log" Jul 21 21:47:13.024307 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:13.024278 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b4hjj/perf-node-gather-daemonset-m96tk" Jul 21 21:47:14.739831 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:14.739752 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694dcbb678-jq7lj_fb9a9560-e187-4512-9f0b-5cdb186f1fab/manager/0.log" Jul 21 21:47:14.763526 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:14.763497 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694dcbb678-jq7lj_fb9a9560-e187-4512-9f0b-5cdb186f1fab/copy-manifests/0.log" Jul 21 21:47:16.411866 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:16.411839 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8f965b796-wnvxm_9f5d702c-52fa-43cb-b7aa-1ac5aa38119a/manager/0.log" Jul 21 21:47:16.677050 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:16.676965 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b46444884-9ztk5_2735e6a0-9b32-480d-9261-865f9df1174a/kube-auth-proxy/0.log" Jul 21 21:47:18.222357 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:18.222328 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b46444884-9ztk5_2735e6a0-9b32-480d-9261-865f9df1174a/kube-auth-proxy/0.log" Jul 21 21:47:23.155129 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:23.155097 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-94p6s_cd4e8142-0742-404d-a5d1-f21f20672947/kube-multus/0.log" Jul 21 21:47:23.574469 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:23.574443 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zqk2d_9bb98f87-4d9c-4654-9e10-e52bfb05591f/kube-multus-additional-cni-plugins/0.log" Jul 21 21:47:23.596354 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:23.596326 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zqk2d_9bb98f87-4d9c-4654-9e10-e52bfb05591f/egress-router-binary-copy/0.log" Jul 21 21:47:23.620983 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:23.620961 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zqk2d_9bb98f87-4d9c-4654-9e10-e52bfb05591f/cni-plugins/0.log" Jul 21 21:47:23.643814 ip-10-0-130-26 kubenswrapper[2568]: I0721 21:47:23.643793 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zqk2d_9bb98f87-4d9c-4654-9e10-e52bfb05591f/bond-cni-plugin/0.log"