Jul 28 17:15:13.516921 ip-10-0-136-215 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 28 17:15:13.516934 ip-10-0-136-215 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 28 17:15:13.516943 ip-10-0-136-215 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 28 17:15:13.517252 ip-10-0-136-215 systemd[1]: Failed to start Kubernetes Kubelet. Jul 28 17:15:23.686488 ip-10-0-136-215 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 28 17:15:23.686508 ip-10-0-136-215 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 01010642cbc940808af1a119f944ecd1 -- Jul 28 17:17:01.452180 ip-10-0-136-215 systemd[1]: Starting Kubernetes Kubelet... Jul 28 17:17:01.835841 ip-10-0-136-215 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:17:01.835841 ip-10-0-136-215 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 17:17:01.835841 ip-10-0-136-215 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:17:01.835841 ip-10-0-136-215 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 17:17:01.835841 ip-10-0-136-215 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:17:01.837622 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.837533 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 17:17:01.840829 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840808 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:01.840829 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840825 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:01.840829 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840831 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:01.840829 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840835 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840839 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840848 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840852 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840857 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840861 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840866 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840870 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840874 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840877 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840881 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840885 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840889 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840892 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840897 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840915 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840920 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840924 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840928 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:01.841086 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840932 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840936 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840943 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840949 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840953 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840957 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840961 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840966 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840970 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840976 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840980 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840984 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840989 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840993 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.840998 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841002 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841007 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841012 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841016 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:01.841709 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841020 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841024 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841028 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841035 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841041 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841046 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841051 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841055 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841059 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841065 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841070 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841074 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841078 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841082 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841086 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841090 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841095 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841099 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841104 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841108 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:01.842244 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841113 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841118 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841122 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841127 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841131 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841135 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841139 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841143 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841148 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841152 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841158 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841162 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841167 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841174 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841179 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841184 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841188 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841193 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841197 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841201 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:01.843112 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841205 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841211 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841216 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841220 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841224 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841229 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841233 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841237 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841241 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841245 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841250 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841254 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841258 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841264 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841268 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841272 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841276 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.841280 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843341 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843352 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:01.843967 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843357 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843372 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843377 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843393 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843398 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843402 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843407 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843411 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843415 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843422 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843429 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843433 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843438 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843443 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843448 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843452 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843457 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843462 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843466 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843470 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:01.844608 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843475 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843479 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843484 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843488 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843492 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843496 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843501 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843505 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843509 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843513 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843518 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843522 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843526 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843533 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843537 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843541 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843547 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843551 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843556 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843560 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:01.845135 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843566 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843572 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843577 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843581 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843585 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843590 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843594 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843598 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843603 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843607 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843614 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843620 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843625 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843629 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843634 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843638 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843649 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843654 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843658 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:01.845736 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843663 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843668 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843673 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843677 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843681 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843686 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843690 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843694 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843698 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843703 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843712 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843716 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843721 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843725 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843729 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843733 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843737 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843742 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843746 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843750 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:01.846240 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843754 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843759 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843763 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843767 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843772 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843776 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843780 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843784 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843788 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843794 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843798 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843802 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843807 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843811 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843816 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843820 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843824 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.843828 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844453 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844465 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:01.846719 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844469 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844472 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844475 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844477 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844481 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844483 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844486 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844489 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844492 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844495 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844498 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844500 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844503 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844505 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844508 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844511 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844514 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844516 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844519 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844522 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:01.847232 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844525 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844527 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844530 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844532 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844535 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844537 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844540 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844542 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844545 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844547 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844550 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844554 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844557 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844560 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844562 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844565 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844568 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844570 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844573 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844575 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:01.847711 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844578 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844583 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844586 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844589 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844592 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844594 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844597 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844600 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844602 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844605 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844608 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844610 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844613 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844616 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844621 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844626 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844628 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844631 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844634 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:01.848213 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844637 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844640 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844643 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844646 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844650 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844653 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844656 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844658 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844661 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844664 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844667 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844669 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844672 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844675 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844677 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844680 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844683 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844685 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844688 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844690 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:01.848674 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844693 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844695 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844698 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844700 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844703 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844705 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844708 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844712 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844716 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844719 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844721 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844724 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844727 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844730 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844732 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844735 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844741 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.844744 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844822 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844833 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844843 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 17:17:01.849167 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844848 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844852 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844856 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844860 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844865 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844868 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844871 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844874 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844877 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844880 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844883 2576 flags.go:64] FLAG: --cgroup-root="" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844886 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844889 2576 flags.go:64] FLAG: --client-ca-file="" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844892 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844895 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844913 2576 flags.go:64] FLAG: --cluster-domain="" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844916 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844919 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844923 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844926 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844930 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844933 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844936 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 17:17:01.849698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844939 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844942 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844945 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844948 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844954 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844957 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844961 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844965 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844968 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844971 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844974 2576 flags.go:64] FLAG: --enable-server="true" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844977 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844982 2576 flags.go:64] FLAG: --event-burst="100" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844985 2576 flags.go:64] FLAG: --event-qps="50" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844988 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844991 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844994 2576 flags.go:64] FLAG: --eviction-hard="" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.844998 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845001 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845004 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845008 2576 flags.go:64] FLAG: --eviction-soft="" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845010 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845013 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845016 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845019 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 17:17:01.850273 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845022 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845025 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845028 2576 flags.go:64] FLAG: --feature-gates="" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845031 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845034 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845037 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845040 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845043 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845047 2576 flags.go:64] FLAG: --help="false" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845049 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-136-215.ec2.internal" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845053 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845057 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845060 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845063 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845067 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845070 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845073 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845076 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845079 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845082 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845085 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845088 2576 flags.go:64] FLAG: --kube-reserved="" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845091 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845093 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 17:17:01.850927 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845096 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845099 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845102 2576 flags.go:64] FLAG: --lock-file="" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845105 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845108 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845111 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845116 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845119 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845121 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845124 2576 flags.go:64] FLAG: --logging-format="text" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845127 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845130 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845135 2576 flags.go:64] FLAG: --manifest-url="" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845138 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845142 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845145 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845150 2576 flags.go:64] FLAG: --max-pods="110" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845153 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845156 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845160 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845163 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845166 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845169 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845172 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845180 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 17:17:01.851534 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845183 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845186 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845189 2576 flags.go:64] FLAG: --pod-cidr="" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845192 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845197 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845199 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845202 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845206 2576 flags.go:64] FLAG: --port="10250" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845209 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845212 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-002dd35cdf827c052" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845215 2576 flags.go:64] FLAG: --qos-reserved="" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845218 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845221 2576 flags.go:64] FLAG: --register-node="true" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845224 2576 flags.go:64] FLAG: --register-with-taints="" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845227 2576 flags.go:64] FLAG: --registry-burst="10" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845230 2576 flags.go:64] FLAG: --registry-qps="5" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845233 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845236 2576 flags.go:64] FLAG: --reserved-memory="" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845239 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845242 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845246 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845249 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845252 2576 flags.go:64] FLAG: --runonce="false" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845255 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845258 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 17:17:01.852157 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845261 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845265 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845268 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845271 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845274 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845277 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845280 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845283 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845286 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845289 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845292 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845295 2576 flags.go:64] FLAG: --system-cgroups="" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845298 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845310 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845313 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845316 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845320 2576 flags.go:64] FLAG: --tls-min-version="" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845323 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845326 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845329 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845332 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845335 2576 flags.go:64] FLAG: --v="2" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845343 2576 flags.go:64] FLAG: --version="false" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845347 2576 flags.go:64] FLAG: --vmodule="" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845350 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 17:17:01.852762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.845353 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845469 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845475 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845478 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845481 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845484 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845487 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845490 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845494 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845497 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845500 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845503 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845507 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845510 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845512 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845515 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845518 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845521 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845523 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845526 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:01.853398 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845529 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845531 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845534 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845537 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845539 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845542 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845544 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845547 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845551 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845554 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845557 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845560 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845563 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845565 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845569 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845572 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845575 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845577 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845580 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:01.853882 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845582 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845586 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845589 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845591 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845594 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845597 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845600 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845604 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845606 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845610 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845613 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845616 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845618 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845621 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845623 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845626 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845628 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845631 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845633 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:01.854367 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845636 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845639 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845641 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845643 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845646 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845649 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845652 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845654 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845658 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845661 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845663 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845666 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845668 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845671 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845675 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845677 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845680 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845683 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845685 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845688 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:01.854856 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845690 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845693 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845695 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845698 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845701 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845704 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845708 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845711 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845714 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845717 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845720 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845722 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845725 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845728 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845730 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845733 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845735 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845738 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845741 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:01.855371 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845743 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845748 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.845751 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.846361 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.852851 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.852869 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.852888 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.852894 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852957 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852963 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852966 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852969 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852972 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852975 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852979 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:01.855837 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852982 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852986 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852989 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852992 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852995 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.852997 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853000 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853002 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853005 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853007 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853010 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853013 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853015 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853018 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853020 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853023 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853025 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853028 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853030 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:01.856226 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853035 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853038 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853041 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853045 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853048 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853051 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853054 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853056 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853059 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853062 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853065 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853067 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853070 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853072 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853075 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853078 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853082 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853085 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853088 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853090 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:01.856681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853093 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853095 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853098 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853101 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853104 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853108 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853111 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853114 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853117 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853119 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853122 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853124 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853127 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853129 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853132 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853136 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853138 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853141 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853143 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853146 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:01.857197 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853148 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853151 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853154 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853157 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853159 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853162 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853164 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853167 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853171 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853174 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853176 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853179 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853182 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853185 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853187 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853190 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853193 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853195 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853198 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853200 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:01.857699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853203 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853205 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853208 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853210 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853213 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853215 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853218 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853221 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853224 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853226 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853229 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853231 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853234 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.853239 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853345 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:01.858275 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853349 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853352 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853355 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853358 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853361 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853365 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853369 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853372 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853375 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853377 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853381 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853384 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853387 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853400 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853403 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853406 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853409 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853411 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853414 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:01.858640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853417 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853419 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853422 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853425 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853427 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853430 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853433 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853435 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853438 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853441 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853443 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853445 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853448 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853451 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853453 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853456 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853458 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853461 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853463 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853466 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:01.859102 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853469 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853472 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853474 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853477 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853479 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853482 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853484 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853486 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853489 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853491 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853494 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853497 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853499 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853502 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853504 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853507 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853509 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853512 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853515 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853517 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:01.859581 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853520 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853522 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853525 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853527 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853529 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853532 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853534 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853537 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853539 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853542 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853544 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853547 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853549 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853552 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853554 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853557 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853559 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853562 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853565 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:01.860072 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853568 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853572 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853575 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853578 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853581 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853583 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853586 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853588 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853591 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853593 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853596 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853598 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853601 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853604 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853606 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853608 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853611 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853613 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853616 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:01.860577 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:01.853618 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:01.861335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.853623 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:17:01.861335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.853712 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 17:17:01.861335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.856271 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 17:17:01.861335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.857510 2576 server.go:1019] "Starting client certificate rotation" Jul 28 17:17:01.861335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.857604 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:17:01.861335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.857630 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:17:01.861335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.859810 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:17:01.863878 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.863856 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:17:01.875221 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.875202 2576 log.go:25] "Validated CRI v1 runtime API" Jul 28 17:17:01.877420 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.877401 2576 log.go:25] "Validated CRI v1 image API" Jul 28 17:17:01.879008 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.878988 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 17:17:01.881544 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.881522 2576 fs.go:135] Filesystem UUIDs: map[209115d9-f8bd-4890-b4f0-de81d8943863:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c1308981-1023-4e05-8a54-429aff3c656a:/dev/nvme0n1p3] Jul 28 17:17:01.881612 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.881542 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 17:17:01.886519 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.886420 2576 manager.go:216] Machine: {Timestamp:2026-07-28 17:17:01.885106678 +0000 UTC m=+0.320537469 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103463 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21feaae4152245e0a3750db06d373d SystemUUID:ec21feaa-e415-2245-e0a3-750db06d373d BootID:01010642-cbc9-4080-8af1-a119f944ecd1 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fd:b9:b3:2f:69 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fd:b9:b3:2f:69 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:53:ba:9e:7a:d3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 17:17:01.886519 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.886512 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 17:17:01.886638 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.886585 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 17:17:01.889386 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.889358 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 17:17:01.889537 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.889396 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-215.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 17:17:01.890006 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.889996 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 17:17:01.890040 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.890009 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 17:17:01.890040 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.890017 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:17:01.890100 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.890085 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:17:01.890668 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.890657 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:17:01.891753 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.891743 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 17:17:01.894665 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.894653 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:17:01.894775 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.894766 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 17:17:01.894806 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.894795 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 28 17:17:01.894835 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.894813 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 17:17:01.894835 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.894830 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 17:17:01.894888 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.894840 2576 kubelet.go:398] "Adding apiserver pod source" Jul 28 17:17:01.894888 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.894849 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 17:17:01.895257 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.895243 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:17:01.895320 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.895262 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:17:01.898006 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.897985 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 17:17:01.899359 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.899346 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 17:17:01.899399 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.899376 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 17:17:01.900062 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900050 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 17:17:01.900118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900066 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 17:17:01.900118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900072 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 17:17:01.900118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900079 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 17:17:01.900118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900088 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 17:17:01.900118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900094 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 17:17:01.900118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900102 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 17:17:01.900118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900107 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 17:17:01.900118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900117 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 17:17:01.900326 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900122 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 17:17:01.900326 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900131 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 17:17:01.900326 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.900136 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 17:17:01.901023 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.901012 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 17:17:01.901057 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.901024 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 17:17:01.904597 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.904584 2576 server.go:1268] "Started kubelet" Jul 28 17:17:01.908554 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.908402 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 17:17:01.909006 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.908986 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-215.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 17:17:01.909213 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:01.909190 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 17:17:01.909414 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:01.909383 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 17:17:01.909498 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.909379 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 17:17:01.909498 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.909481 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 17:17:01.909428 ip-10-0-136-215 systemd[1]: Started Kubernetes Kubelet. Jul 28 17:17:01.912347 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.912302 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 28 17:17:01.913279 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.913263 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 17:17:01.914345 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:01.914319 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 17:17:01.914738 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:01.913863 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-215.ec2.internal.18c6839f28ae6756 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-215.ec2.internal,UID:ip-10-0-136-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-215.ec2.internal,},FirstTimestamp:2026-07-28 17:17:01.90453743 +0000 UTC m=+0.339968219,LastTimestamp:2026-07-28 17:17:01.90453743 +0000 UTC m=+0.339968219,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-215.ec2.internal,}" Jul 28 17:17:01.917845 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.917826 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 17:17:01.917845 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.917839 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 17:17:01.918341 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:01.918138 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:01.918423 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.918395 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 17:17:01.918503 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.918485 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 17:17:01.918572 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.918564 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 17:17:01.918768 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.918753 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 28 17:17:01.918768 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.918768 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 28 17:17:01.918883 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.918769 2576 factory.go:55] Registering systemd factory Jul 28 17:17:01.918883 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.918816 2576 factory.go:223] Registration of the systemd container factory successfully Jul 28 17:17:01.919097 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.919077 2576 factory.go:153] Registering CRI-O factory Jul 28 17:17:01.919097 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.919095 2576 factory.go:223] Registration of the crio container factory successfully Jul 28 17:17:01.919234 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.919145 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 17:17:01.919234 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.919229 2576 factory.go:103] Registering Raw factory Jul 28 17:17:01.919401 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.919246 2576 manager.go:1196] Started watching for new ooms in manager Jul 28 17:17:01.919601 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.919588 2576 manager.go:349] Starting recovery of all containers Jul 28 17:17:01.920103 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:01.920077 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 17:17:01.920188 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:01.920170 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 17:17:01.925073 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.925052 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bfr6q" Jul 28 17:17:01.928287 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.928247 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 17:17:01.929587 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.929568 2576 manager.go:354] Recovery completed Jul 28 17:17:01.933117 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.933101 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bfr6q" Jul 28 17:17:01.933969 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.933955 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:01.935994 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.935976 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:01.936059 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.936007 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:01.936059 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.936030 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:01.936828 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.936811 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 17:17:01.936939 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.936832 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 17:17:01.936939 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.936849 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:17:01.938370 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.938354 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:01.938738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.938721 2576 policy_none.go:49] "None policy: Start" Jul 28 17:17:01.938813 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:01.938618 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-215.ec2.internal.18c6839f2a8e6373 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-215.ec2.internal,UID:ip-10-0-136-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-215.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-215.ec2.internal,},FirstTimestamp:2026-07-28 17:17:01.935993715 +0000 UTC m=+0.371424504,LastTimestamp:2026-07-28 17:17:01.935993715 +0000 UTC m=+0.371424504,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-215.ec2.internal,}" Jul 28 17:17:01.938813 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.938755 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 17:17:01.939053 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.939036 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 17:17:01.939493 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.939474 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:01.939576 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.939502 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:01.939576 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.939512 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:01.940352 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.940338 2576 policy_none.go:47] "Start" Jul 28 17:17:01.970660 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.970548 2576 manager.go:337] "Starting Device Plugin manager" Jul 28 17:17:01.970741 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:01.970682 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 17:17:01.970741 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.970696 2576 server.go:87] "Starting device plugin registration server" Jul 28 17:17:01.970947 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.970934 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 17:17:01.970978 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.970952 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 17:17:01.971048 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.971032 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 17:17:01.971195 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.971120 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 17:17:01.971195 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.971132 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 17:17:01.971735 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:01.971713 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 17:17:01.971804 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:01.971768 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:01.989493 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:01.989468 2576 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2615 into the system container \"/system.slice\": " Jul 28 17:17:02.071390 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.071357 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:02.072464 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.072239 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:02.072464 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.072270 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:02.072464 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.072280 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:02.072464 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.072304 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.074698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.074676 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 17:17:02.074782 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.074711 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 17:17:02.074782 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.074729 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 17:17:02.074782 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.074757 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 17:17:02.076732 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.076709 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:17:02.081505 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.081488 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.081575 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.081513 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-215.ec2.internal\": node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:02.109715 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.109695 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:02.175158 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.175129 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal"] Jul 28 17:17:02.175251 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.175188 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:02.176479 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.176456 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:02.176562 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.176485 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:02.176562 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.176495 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:02.177558 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.177546 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:02.177675 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.177661 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.177734 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.177694 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:02.178216 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.178202 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:02.178298 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.178225 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:02.178298 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.178237 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:02.178298 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.178248 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:02.178298 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.178268 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:02.178298 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.178278 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:02.179286 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.179272 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.179329 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.179296 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:02.179915 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.179887 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:02.179996 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.179930 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:02.179996 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.179945 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:02.193441 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.193419 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-215.ec2.internal\" not found" node="ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.197124 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.197111 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-215.ec2.internal\" not found" node="ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.210142 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.210122 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:02.220512 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.220491 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.220590 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.220519 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.220590 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.220544 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4746d2d48c5a0af4bf6819c4fa3b9ae9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-215.ec2.internal\" (UID: \"4746d2d48c5a0af4bf6819c4fa3b9ae9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.311180 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.311159 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:02.321559 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.321538 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4746d2d48c5a0af4bf6819c4fa3b9ae9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-215.ec2.internal\" (UID: \"4746d2d48c5a0af4bf6819c4fa3b9ae9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.321621 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.321588 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4746d2d48c5a0af4bf6819c4fa3b9ae9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-215.ec2.internal\" (UID: \"4746d2d48c5a0af4bf6819c4fa3b9ae9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.321659 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.321629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.321659 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.321652 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.321720 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.321688 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.321751 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.321720 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.411997 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.411937 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:02.495568 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.495522 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.500774 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.500760 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:17:02.512708 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.512684 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:02.613247 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.613209 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:02.713813 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.713739 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:02.759293 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.759273 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:17:02.813893 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.813866 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:02.856957 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.856936 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 17:17:02.857324 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.857105 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 17:17:02.857324 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:02.857108 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 17:17:02.914088 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:02.914071 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:03.999442 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:03.999403 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:12:02 +0000 UTC" deadline="2028-03-04 12:36:42.096896346 +0000 UTC" Jul 28 17:17:03.999442 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:03.999441 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14035h19m38.09745754s" Jul 28 17:17:04.077764 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.077740 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:04.086446 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:04.086421 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42785eed679f9257ac8637d2df48ef77.slice/crio-8bcb96e24868f89e031108017fb1361778f44b01ca8d29c3517871bf8e5ce269 WatchSource:0}: Error finding container 8bcb96e24868f89e031108017fb1361778f44b01ca8d29c3517871bf8e5ce269: Status 404 returned error can't find the container with id 8bcb96e24868f89e031108017fb1361778f44b01ca8d29c3517871bf8e5ce269 Jul 28 17:17:04.090842 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.090829 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:17:04.111171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.111152 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:17:04.142071 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.142031 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" event={"ID":"42785eed679f9257ac8637d2df48ef77","Type":"ContainerStarted","Data":"8bcb96e24868f89e031108017fb1361778f44b01ca8d29c3517871bf8e5ce269"} Jul 28 17:17:04.161479 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:04.161452 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4746d2d48c5a0af4bf6819c4fa3b9ae9.slice/crio-6a27545d34170bf5b178247207bb40d5db8a17f306e9c38a5c52d3f240ee640a WatchSource:0}: Error finding container 6a27545d34170bf5b178247207bb40d5db8a17f306e9c38a5c52d3f240ee640a: Status 404 returned error can't find the container with id 6a27545d34170bf5b178247207bb40d5db8a17f306e9c38a5c52d3f240ee640a Jul 28 17:17:04.178014 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.177999 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:04.278627 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.278600 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:04.379145 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.379092 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:04.479909 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.479886 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:04.580449 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.580410 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:04.680910 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.680833 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:04.781358 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.781319 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:04.881837 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.881802 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:17:04.922878 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.922850 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:17:04.959293 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.959214 2576 apiserver.go:52] "Watching apiserver" Jul 28 17:17:04.967651 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.967620 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 17:17:04.969806 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.969760 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-fbcx8","openshift-ovn-kubernetes/ovnkube-node-c29m8","kube-system/global-pull-secret-syncer-tjmr6","openshift-cluster-node-tuning-operator/tuned-w5dwp","openshift-image-registry/node-ca-87p9h","openshift-multus/multus-additional-cni-plugins-l8f82","kube-system/konnectivity-agent-xtm4c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr","openshift-dns/node-resolver-4vvsd","openshift-multus/multus-flxnq","openshift-multus/network-metrics-daemon-s9qjh","openshift-network-diagnostics/network-check-target-5g2mj"] Jul 28 17:17:04.971427 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.971294 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fbcx8" Jul 28 17:17:04.973012 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.972987 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.973961 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.973913 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-p5czz\"" Jul 28 17:17:04.973961 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.973938 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 17:17:04.973961 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.973936 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 17:17:04.974163 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.973937 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:17:04.974549 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.974332 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:04.974549 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.974441 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:04.975855 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.975517 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.975855 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.975723 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gs794\"" Jul 28 17:17:04.977415 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.976956 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 17:17:04.977415 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.977214 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 17:17:04.977415 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.977238 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 17:17:04.977415 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.977242 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 17:17:04.977415 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.977347 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 17:17:04.977875 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.977472 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 17:17:04.977875 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.977643 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87p9h" Jul 28 17:17:04.977875 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.977835 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 17:17:04.978135 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.978110 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sshjh\"" Jul 28 17:17:04.978135 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.978131 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:04.978247 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.978150 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:17:04.980859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.980837 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 17:17:04.981279 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.981158 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 17:17:04.981279 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.981185 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:17:04.981279 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.981261 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 17:17:04.981279 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.981280 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8769f\"" Jul 28 17:17:04.983668 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.981969 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 17:17:04.983668 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.982099 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 17:17:04.983668 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.982231 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t6sq4\"" Jul 28 17:17:04.983668 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.982348 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 17:17:04.983668 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.982408 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 17:17:04.983668 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.982559 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 17:17:04.983668 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.982700 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 17:17:04.985535 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.985513 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:04.987112 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.987091 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:04.987871 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.987708 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 17:17:04.987871 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.987750 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cwk8s\"" Jul 28 17:17:04.988014 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.987909 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 17:17:04.988227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.988211 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4vvsd" Jul 28 17:17:04.989564 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.989518 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 17:17:04.989749 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.989733 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 17:17:04.989869 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.989837 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-flxnq" Jul 28 17:17:04.989869 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.989854 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6hp24\"" Jul 28 17:17:04.990238 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.990217 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 17:17:04.990416 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.990398 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hw2rj\"" Jul 28 17:17:04.990686 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.990672 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 17:17:04.990752 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.990690 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 17:17:04.991161 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.991138 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:04.991246 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.991208 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:04.992204 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.992168 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 17:17:04.992322 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.992296 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rchxq\"" Jul 28 17:17:04.993130 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993106 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c71f50e-2184-425b-9d24-0e2c2aebcaac-kubelet-config\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:04.993235 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993142 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-sysconfig\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.993235 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993159 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-sys\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.993235 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993172 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-host\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.993235 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993214 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c13cd9b-abcc-40c7-98f1-15d3b843c740-tmp\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.993445 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993251 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-cnibin\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:04.993445 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993296 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-run-openvswitch\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.993445 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993319 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-node-log\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.993445 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993356 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:04.993445 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993380 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d-agent-certs\") pod \"konnectivity-agent-xtm4c\" (UID: \"de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d\") " pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:04.993445 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993404 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12f6882d-5ac2-4ab2-b01f-b295d9eba005-host-slash\") pod \"iptables-alerter-fbcx8\" (UID: \"12f6882d-5ac2-4ab2-b01f-b295d9eba005\") " pod="openshift-network-operator/iptables-alerter-fbcx8" Jul 28 17:17:04.993445 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993426 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-slash\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993457 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-cni-netd\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993483 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54b22a64-9b05-4404-be84-b23e2c9d6af6-ovnkube-config\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993505 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:04.993567 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993511 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c795a38b-13c3-4b44-ae37-9de0fd450a00-host\") pod \"node-ca-87p9h\" (UID: \"c795a38b-13c3-4b44-ae37-9de0fd450a00\") " pod="openshift-image-registry/node-ca-87p9h" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993647 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c71f50e-2184-425b-9d24-0e2c2aebcaac-dbus\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993683 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-sysctl-conf\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993697 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-systemd\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993714 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-tuned\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993737 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed55c986-c239-4f25-a72f-539854fa246d-cni-binary-copy\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993829 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d-konnectivity-ca\") pod \"konnectivity-agent-xtm4c\" (UID: \"de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d\") " pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:04.993859 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993856 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-modprobe-d\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993877 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-kubernetes\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993889 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993902 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-run\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993938 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-var-lib-kubelet\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.993963 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994011 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed55c986-c239-4f25-a72f-539854fa246d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994046 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc9vw\" (UniqueName: \"kubernetes.io/projected/12f6882d-5ac2-4ab2-b01f-b295d9eba005-kube-api-access-qc9vw\") pod \"iptables-alerter-fbcx8\" (UID: \"12f6882d-5ac2-4ab2-b01f-b295d9eba005\") " pod="openshift-network-operator/iptables-alerter-fbcx8" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994073 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed55c986-c239-4f25-a72f-539854fa246d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994107 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994105 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/12f6882d-5ac2-4ab2-b01f-b295d9eba005-iptables-alerter-script\") pod \"iptables-alerter-fbcx8\" (UID: \"12f6882d-5ac2-4ab2-b01f-b295d9eba005\") " pod="openshift-network-operator/iptables-alerter-fbcx8" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994211 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-log-socket\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994240 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-run-ovn-kubernetes\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994182 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal"] Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994264 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-cni-bin\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994288 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994313 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54b22a64-9b05-4404-be84-b23e2c9d6af6-env-overrides\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.994372 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994350 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-lib-modules\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994380 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-system-cni-dir\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994397 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v2t2\" (UniqueName: \"kubernetes.io/projected/ed55c986-c239-4f25-a72f-539854fa246d-kube-api-access-4v2t2\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994425 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-systemd-units\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994454 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-run-netns\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994588 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-run-systemd\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994618 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-etc-openvswitch\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994663 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-run-ovn\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994681 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-sysctl-d\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994695 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlxmp\" (UniqueName: \"kubernetes.io/projected/0c13cd9b-abcc-40c7-98f1-15d3b843c740-kube-api-access-mlxmp\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994714 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-os-release\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994750 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-kubelet\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994765 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54b22a64-9b05-4404-be84-b23e2c9d6af6-ovn-node-metrics-cert\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994779 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pdw9\" (UniqueName: \"kubernetes.io/projected/54b22a64-9b05-4404-be84-b23e2c9d6af6-kube-api-access-5pdw9\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994856 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-var-lib-openvswitch\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994880 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54b22a64-9b05-4404-be84-b23e2c9d6af6-ovnkube-script-lib\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:04.995176 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994959 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c795a38b-13c3-4b44-ae37-9de0fd450a00-serviceca\") pod \"node-ca-87p9h\" (UID: \"c795a38b-13c3-4b44-ae37-9de0fd450a00\") " pod="openshift-image-registry/node-ca-87p9h" Jul 28 17:17:04.995951 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.994993 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75qlx\" (UniqueName: \"kubernetes.io/projected/c795a38b-13c3-4b44-ae37-9de0fd450a00-kube-api-access-75qlx\") pod \"node-ca-87p9h\" (UID: \"c795a38b-13c3-4b44-ae37-9de0fd450a00\") " pod="openshift-image-registry/node-ca-87p9h" Jul 28 17:17:04.996448 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:04.996425 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:17:05.005718 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.005700 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:17:05.006293 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.006276 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal"] Jul 28 17:17:05.022924 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.022907 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-z56pk" Jul 28 17:17:05.033674 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.033629 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-z56pk" Jul 28 17:17:05.082347 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.082321 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 17:17:05.096008 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.095984 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.096164 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096014 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-run-ovn-kubernetes\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.096164 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096039 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-cni-bin\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.096164 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096065 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-socket-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.096164 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096139 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-run-ovn-kubernetes\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.096355 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096168 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-system-cni-dir\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.096355 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096197 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/12f6882d-5ac2-4ab2-b01f-b295d9eba005-iptables-alerter-script\") pod \"iptables-alerter-fbcx8\" (UID: \"12f6882d-5ac2-4ab2-b01f-b295d9eba005\") " pod="openshift-network-operator/iptables-alerter-fbcx8" Jul 28 17:17:05.096355 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096205 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-cni-bin\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.096355 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096219 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-log-socket\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.096355 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096258 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-log-socket\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.096355 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096260 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-daemon-config\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.096355 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096261 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.096355 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096302 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-lib-modules\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.096714 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096397 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-systemd-units\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.096714 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096433 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-run-netns\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.096714 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096455 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-run-ovn\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.096714 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096487 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-run-ovn\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.096714 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096543 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-run-netns\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.096714 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096553 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-lib-modules\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.096714 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096555 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-var-lib-cni-bin\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.096714 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096608 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-hostroot\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.096714 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096646 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-conf-dir\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.096714 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096674 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-sysctl-d\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.096714 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096704 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-kubelet\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096740 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54b22a64-9b05-4404-be84-b23e2c9d6af6-ovn-node-metrics-cert\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096764 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc4np\" (UniqueName: \"kubernetes.io/projected/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-kube-api-access-fc4np\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096799 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-sysctl-d\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096805 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-kubelet\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096805 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j66z7\" (UniqueName: \"kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7\") pod \"network-check-target-5g2mj\" (UID: \"4701c1b9-f966-41c6-bd60-86d9fc2bf11e\") " pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096853 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-systemd-units\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096863 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/12f6882d-5ac2-4ab2-b01f-b295d9eba005-iptables-alerter-script\") pod \"iptables-alerter-fbcx8\" (UID: \"12f6882d-5ac2-4ab2-b01f-b295d9eba005\") " pod="openshift-network-operator/iptables-alerter-fbcx8" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096914 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-cnibin\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096943 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.096983 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c71f50e-2184-425b-9d24-0e2c2aebcaac-kubelet-config\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097006 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-sys\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097028 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-host\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097049 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-node-log\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097071 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-cni-binary-copy\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097094 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-etc-kubernetes\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097098 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c71f50e-2184-425b-9d24-0e2c2aebcaac-kubelet-config\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:05.097171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097115 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-node-log\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097119 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c795a38b-13c3-4b44-ae37-9de0fd450a00-host\") pod \"node-ca-87p9h\" (UID: \"c795a38b-13c3-4b44-ae37-9de0fd450a00\") " pod="openshift-image-registry/node-ca-87p9h" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097112 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-host\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097153 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-sys\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097166 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c795a38b-13c3-4b44-ae37-9de0fd450a00-host\") pod \"node-ca-87p9h\" (UID: \"c795a38b-13c3-4b44-ae37-9de0fd450a00\") " pod="openshift-image-registry/node-ca-87p9h" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097175 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-cni-dir\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097151 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097210 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed55c986-c239-4f25-a72f-539854fa246d-cni-binary-copy\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097249 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-device-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097313 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnsg2\" (UniqueName: \"kubernetes.io/projected/e761eac4-98d4-494a-be69-82b2f0e6e4d4-kube-api-access-vnsg2\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097343 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-modprobe-d\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097385 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-kubernetes\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097412 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-run\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097433 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-var-lib-kubelet\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097453 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-modprobe-d\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097470 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed55c986-c239-4f25-a72f-539854fa246d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097476 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-kubernetes\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097502 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-var-lib-kubelet\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.097977 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097506 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc9vw\" (UniqueName: \"kubernetes.io/projected/12f6882d-5ac2-4ab2-b01f-b295d9eba005-kube-api-access-qc9vw\") pod \"iptables-alerter-fbcx8\" (UID: \"12f6882d-5ac2-4ab2-b01f-b295d9eba005\") " pod="openshift-network-operator/iptables-alerter-fbcx8" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097534 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097545 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-run\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097561 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54b22a64-9b05-4404-be84-b23e2c9d6af6-env-overrides\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097586 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed55c986-c239-4f25-a72f-539854fa246d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097613 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54b22a64-9b05-4404-be84-b23e2c9d6af6-ovnkube-script-lib\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097624 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097640 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-registration-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097664 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-sys-fs\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097688 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scmb8\" (UniqueName: \"kubernetes.io/projected/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-kube-api-access-scmb8\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097711 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-socket-dir-parent\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097727 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed55c986-c239-4f25-a72f-539854fa246d-cni-binary-copy\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097734 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-var-lib-kubelet\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097759 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-system-cni-dir\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097803 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v2t2\" (UniqueName: \"kubernetes.io/projected/ed55c986-c239-4f25-a72f-539854fa246d-kube-api-access-4v2t2\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097864 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-system-cni-dir\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.098738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097874 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-run-systemd\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097910 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-etc-openvswitch\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097934 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-os-release\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097959 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-run-k8s-cni-cncf-io\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.097987 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-run-netns\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098006 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed55c986-c239-4f25-a72f-539854fa246d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098025 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/09009b39-09ab-475e-bc10-1c0f559e4a07-hosts-file\") pod \"node-resolver-4vvsd\" (UID: \"09009b39-09ab-475e-bc10-1c0f559e4a07\") " pod="openshift-dns/node-resolver-4vvsd" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098050 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlxmp\" (UniqueName: \"kubernetes.io/projected/0c13cd9b-abcc-40c7-98f1-15d3b843c740-kube-api-access-mlxmp\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098063 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-run-systemd\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098073 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-os-release\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098085 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed55c986-c239-4f25-a72f-539854fa246d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098098 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pdw9\" (UniqueName: \"kubernetes.io/projected/54b22a64-9b05-4404-be84-b23e2c9d6af6-kube-api-access-5pdw9\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098122 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-etc-openvswitch\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098138 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54b22a64-9b05-4404-be84-b23e2c9d6af6-env-overrides\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098170 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-os-release\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098252 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-etc-selinux\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098290 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09009b39-09ab-475e-bc10-1c0f559e4a07-tmp-dir\") pod \"node-resolver-4vvsd\" (UID: \"09009b39-09ab-475e-bc10-1c0f559e4a07\") " pod="openshift-dns/node-resolver-4vvsd" Jul 28 17:17:05.099542 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098329 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-var-lib-openvswitch\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098378 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c795a38b-13c3-4b44-ae37-9de0fd450a00-serviceca\") pod \"node-ca-87p9h\" (UID: \"c795a38b-13c3-4b44-ae37-9de0fd450a00\") " pod="openshift-image-registry/node-ca-87p9h" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098399 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-var-lib-openvswitch\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098441 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75qlx\" (UniqueName: \"kubernetes.io/projected/c795a38b-13c3-4b44-ae37-9de0fd450a00-kube-api-access-75qlx\") pod \"node-ca-87p9h\" (UID: \"c795a38b-13c3-4b44-ae37-9de0fd450a00\") " pod="openshift-image-registry/node-ca-87p9h" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098470 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098557 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54b22a64-9b05-4404-be84-b23e2c9d6af6-ovnkube-script-lib\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098596 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npgmp\" (UniqueName: \"kubernetes.io/projected/09009b39-09ab-475e-bc10-1c0f559e4a07-kube-api-access-npgmp\") pod \"node-resolver-4vvsd\" (UID: \"09009b39-09ab-475e-bc10-1c0f559e4a07\") " pod="openshift-dns/node-resolver-4vvsd" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098621 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-sysconfig\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098645 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c13cd9b-abcc-40c7-98f1-15d3b843c740-tmp\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098668 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-cnibin\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098692 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-run-openvswitch\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098692 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-sysconfig\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098717 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098759 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed55c986-c239-4f25-a72f-539854fa246d-cnibin\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098760 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d-agent-certs\") pod \"konnectivity-agent-xtm4c\" (UID: \"de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d\") " pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.098807 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098816 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12f6882d-5ac2-4ab2-b01f-b295d9eba005-host-slash\") pod \"iptables-alerter-fbcx8\" (UID: \"12f6882d-5ac2-4ab2-b01f-b295d9eba005\") " pod="openshift-network-operator/iptables-alerter-fbcx8" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098840 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-slash\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.100377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098846 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c795a38b-13c3-4b44-ae37-9de0fd450a00-serviceca\") pod \"node-ca-87p9h\" (UID: \"c795a38b-13c3-4b44-ae37-9de0fd450a00\") " pod="openshift-image-registry/node-ca-87p9h" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.098864 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret podName:6c71f50e-2184-425b-9d24-0e2c2aebcaac nodeName:}" failed. No retries permitted until 2026-07-28 17:17:05.598844979 +0000 UTC m=+2.971393554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret") pod "global-pull-secret-syncer-tjmr6" (UID: "6c71f50e-2184-425b-9d24-0e2c2aebcaac") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098873 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-slash\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098890 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-cni-netd\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098906 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12f6882d-5ac2-4ab2-b01f-b295d9eba005-host-slash\") pod \"iptables-alerter-fbcx8\" (UID: \"12f6882d-5ac2-4ab2-b01f-b295d9eba005\") " pod="openshift-network-operator/iptables-alerter-fbcx8" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098916 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54b22a64-9b05-4404-be84-b23e2c9d6af6-ovnkube-config\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098941 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-run-multus-certs\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098964 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c71f50e-2184-425b-9d24-0e2c2aebcaac-dbus\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.098985 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-sysctl-conf\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.099009 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-systemd\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.099031 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-tuned\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.099053 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d-konnectivity-ca\") pod \"konnectivity-agent-xtm4c\" (UID: \"de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d\") " pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.099079 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-var-lib-cni-multus\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.099167 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c71f50e-2184-425b-9d24-0e2c2aebcaac-dbus\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.099206 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-host-cni-netd\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.099225 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-sysctl-conf\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.099274 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-systemd\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.101227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.099307 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b22a64-9b05-4404-be84-b23e2c9d6af6-run-openvswitch\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.102014 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.099694 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54b22a64-9b05-4404-be84-b23e2c9d6af6-ovnkube-config\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.102014 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.099847 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d-konnectivity-ca\") pod \"konnectivity-agent-xtm4c\" (UID: \"de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d\") " pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:05.102014 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.101264 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54b22a64-9b05-4404-be84-b23e2c9d6af6-ovn-node-metrics-cert\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.102014 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.101418 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0c13cd9b-abcc-40c7-98f1-15d3b843c740-etc-tuned\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.102014 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.101717 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d-agent-certs\") pod \"konnectivity-agent-xtm4c\" (UID: \"de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d\") " pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:05.102014 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.101796 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c13cd9b-abcc-40c7-98f1-15d3b843c740-tmp\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.107423 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.107397 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v2t2\" (UniqueName: \"kubernetes.io/projected/ed55c986-c239-4f25-a72f-539854fa246d-kube-api-access-4v2t2\") pod \"multus-additional-cni-plugins-l8f82\" (UID: \"ed55c986-c239-4f25-a72f-539854fa246d\") " pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.107533 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.107431 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pdw9\" (UniqueName: \"kubernetes.io/projected/54b22a64-9b05-4404-be84-b23e2c9d6af6-kube-api-access-5pdw9\") pod \"ovnkube-node-c29m8\" (UID: \"54b22a64-9b05-4404-be84-b23e2c9d6af6\") " pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.108195 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.108146 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlxmp\" (UniqueName: \"kubernetes.io/projected/0c13cd9b-abcc-40c7-98f1-15d3b843c740-kube-api-access-mlxmp\") pod \"tuned-w5dwp\" (UID: \"0c13cd9b-abcc-40c7-98f1-15d3b843c740\") " pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.108608 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.108586 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75qlx\" (UniqueName: \"kubernetes.io/projected/c795a38b-13c3-4b44-ae37-9de0fd450a00-kube-api-access-75qlx\") pod \"node-ca-87p9h\" (UID: \"c795a38b-13c3-4b44-ae37-9de0fd450a00\") " pod="openshift-image-registry/node-ca-87p9h" Jul 28 17:17:05.109166 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.109147 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc9vw\" (UniqueName: \"kubernetes.io/projected/12f6882d-5ac2-4ab2-b01f-b295d9eba005-kube-api-access-qc9vw\") pod \"iptables-alerter-fbcx8\" (UID: \"12f6882d-5ac2-4ab2-b01f-b295d9eba005\") " pod="openshift-network-operator/iptables-alerter-fbcx8" Jul 28 17:17:05.144915 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.144876 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" event={"ID":"4746d2d48c5a0af4bf6819c4fa3b9ae9","Type":"ContainerStarted","Data":"6a27545d34170bf5b178247207bb40d5db8a17f306e9c38a5c52d3f240ee640a"} Jul 28 17:17:05.199460 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199433 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-run-multus-certs\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199460 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199464 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-var-lib-cni-multus\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199644 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199481 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-socket-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.199644 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199503 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-system-cni-dir\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199644 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199513 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-run-multus-certs\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199644 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199621 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-var-lib-cni-multus\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199657 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-daemon-config\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199693 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-var-lib-cni-bin\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199709 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-system-cni-dir\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199719 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-hostroot\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199753 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-hostroot\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199760 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-conf-dir\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199796 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc4np\" (UniqueName: \"kubernetes.io/projected/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-kube-api-access-fc4np\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199802 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-var-lib-cni-bin\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.199867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199659 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-socket-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.199867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199824 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j66z7\" (UniqueName: \"kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7\") pod \"network-check-target-5g2mj\" (UID: \"4701c1b9-f966-41c6-bd60-86d9fc2bf11e\") " pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:05.199867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199800 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-conf-dir\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.199973 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-cnibin\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200015 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200023 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-cnibin\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200048 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-cni-binary-copy\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200071 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-etc-kubernetes\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200097 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-cni-dir\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.200101 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200122 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-device-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200110 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-etc-kubernetes\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200146 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnsg2\" (UniqueName: \"kubernetes.io/projected/e761eac4-98d4-494a-be69-82b2f0e6e4d4-kube-api-access-vnsg2\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.200164 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs podName:e761eac4-98d4-494a-be69-82b2f0e6e4d4 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:05.700146956 +0000 UTC m=+3.072695529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs") pod "network-metrics-daemon-s9qjh" (UID: "e761eac4-98d4-494a-be69-82b2f0e6e4d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200190 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-daemon-config\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200205 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-cni-dir\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200206 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-device-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200256 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-registration-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200290 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-sys-fs\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.200385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scmb8\" (UniqueName: \"kubernetes.io/projected/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-kube-api-access-scmb8\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200340 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-socket-dir-parent\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200366 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-var-lib-kubelet\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200371 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-sys-fs\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200392 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-os-release\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200415 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-run-k8s-cni-cncf-io\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200416 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-var-lib-kubelet\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200389 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-registration-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200432 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-multus-socket-dir-parent\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200440 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-run-netns\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200470 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-run-netns\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200474 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-cni-binary-copy\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200482 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-os-release\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200476 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-host-run-k8s-cni-cncf-io\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200470 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/09009b39-09ab-475e-bc10-1c0f559e4a07-hosts-file\") pod \"node-resolver-4vvsd\" (UID: \"09009b39-09ab-475e-bc10-1c0f559e4a07\") " pod="openshift-dns/node-resolver-4vvsd" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200505 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/09009b39-09ab-475e-bc10-1c0f559e4a07-hosts-file\") pod \"node-resolver-4vvsd\" (UID: \"09009b39-09ab-475e-bc10-1c0f559e4a07\") " pod="openshift-dns/node-resolver-4vvsd" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200542 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-etc-selinux\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200566 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09009b39-09ab-475e-bc10-1c0f559e4a07-tmp-dir\") pod \"node-resolver-4vvsd\" (UID: \"09009b39-09ab-475e-bc10-1c0f559e4a07\") " pod="openshift-dns/node-resolver-4vvsd" Jul 28 17:17:05.201124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.201687 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200624 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npgmp\" (UniqueName: \"kubernetes.io/projected/09009b39-09ab-475e-bc10-1c0f559e4a07-kube-api-access-npgmp\") pod \"node-resolver-4vvsd\" (UID: \"09009b39-09ab-475e-bc10-1c0f559e4a07\") " pod="openshift-dns/node-resolver-4vvsd" Jul 28 17:17:05.201687 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200632 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-etc-selinux\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.201687 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200711 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.201687 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.200918 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09009b39-09ab-475e-bc10-1c0f559e4a07-tmp-dir\") pod \"node-resolver-4vvsd\" (UID: \"09009b39-09ab-475e-bc10-1c0f559e4a07\") " pod="openshift-dns/node-resolver-4vvsd" Jul 28 17:17:05.207969 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.207946 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:05.207969 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.207972 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:05.208158 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.207987 2576 projected.go:196] Error preparing data for projected volume kube-api-access-j66z7 for pod openshift-network-diagnostics/network-check-target-5g2mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:05.208158 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.208051 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7 podName:4701c1b9-f966-41c6-bd60-86d9fc2bf11e nodeName:}" failed. No retries permitted until 2026-07-28 17:17:05.708032873 +0000 UTC m=+3.080581447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j66z7" (UniqueName: "kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7") pod "network-check-target-5g2mj" (UID: "4701c1b9-f966-41c6-bd60-86d9fc2bf11e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:05.210885 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.210604 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc4np\" (UniqueName: \"kubernetes.io/projected/bc2a32ca-505c-45a6-8ab0-8b994aa479f0-kube-api-access-fc4np\") pod \"multus-flxnq\" (UID: \"bc2a32ca-505c-45a6-8ab0-8b994aa479f0\") " pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.210885 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.210659 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scmb8\" (UniqueName: \"kubernetes.io/projected/2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5-kube-api-access-scmb8\") pod \"aws-ebs-csi-driver-node-wvwzr\" (UID: \"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.210885 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.210838 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npgmp\" (UniqueName: \"kubernetes.io/projected/09009b39-09ab-475e-bc10-1c0f559e4a07-kube-api-access-npgmp\") pod \"node-resolver-4vvsd\" (UID: \"09009b39-09ab-475e-bc10-1c0f559e4a07\") " pod="openshift-dns/node-resolver-4vvsd" Jul 28 17:17:05.211133 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.211116 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnsg2\" (UniqueName: \"kubernetes.io/projected/e761eac4-98d4-494a-be69-82b2f0e6e4d4-kube-api-access-vnsg2\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:05.289197 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.289164 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fbcx8" Jul 28 17:17:05.296500 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.296477 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:05.304944 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.304925 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" Jul 28 17:17:05.309281 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.309221 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87p9h" Jul 28 17:17:05.316640 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.316618 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l8f82" Jul 28 17:17:05.322417 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.322302 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:05.330413 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.330393 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" Jul 28 17:17:05.335501 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.335483 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4vvsd" Jul 28 17:17:05.340594 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.340576 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-flxnq" Jul 28 17:17:05.493266 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.493179 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:17:05.602869 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.602839 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:05.603030 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.602995 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:05.603074 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.603052 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret podName:6c71f50e-2184-425b-9d24-0e2c2aebcaac nodeName:}" failed. No retries permitted until 2026-07-28 17:17:06.603037927 +0000 UTC m=+3.975586501 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret") pod "global-pull-secret-syncer-tjmr6" (UID: "6c71f50e-2184-425b-9d24-0e2c2aebcaac") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:05.703738 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.703708 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:05.703938 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.703863 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:05.703938 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.703923 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs podName:e761eac4-98d4-494a-be69-82b2f0e6e4d4 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:06.703906806 +0000 UTC m=+4.076455384 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs") pod "network-metrics-daemon-s9qjh" (UID: "e761eac4-98d4-494a-be69-82b2f0e6e4d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:05.804757 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:05.804662 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j66z7\" (UniqueName: \"kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7\") pod \"network-check-target-5g2mj\" (UID: \"4701c1b9-f966-41c6-bd60-86d9fc2bf11e\") " pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:05.804936 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.804852 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:05.804936 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.804882 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:05.804936 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.804895 2576 projected.go:196] Error preparing data for projected volume kube-api-access-j66z7 for pod openshift-network-diagnostics/network-check-target-5g2mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:05.805098 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:05.804948 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7 podName:4701c1b9-f966-41c6-bd60-86d9fc2bf11e nodeName:}" failed. No retries permitted until 2026-07-28 17:17:06.804931428 +0000 UTC m=+4.177480074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j66z7" (UniqueName: "kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7") pod "network-check-target-5g2mj" (UID: "4701c1b9-f966-41c6-bd60-86d9fc2bf11e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:05.945248 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:05.945216 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12f6882d_5ac2_4ab2_b01f_b295d9eba005.slice/crio-3f7134622e4a9b7cd85f9f7a3c85fb515c3adfc0aaa487c5feb01a80d484e845 WatchSource:0}: Error finding container 3f7134622e4a9b7cd85f9f7a3c85fb515c3adfc0aaa487c5feb01a80d484e845: Status 404 returned error can't find the container with id 3f7134622e4a9b7cd85f9f7a3c85fb515c3adfc0aaa487c5feb01a80d484e845 Jul 28 17:17:05.947396 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:05.947367 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded55c986_c239_4f25_a72f_539854fa246d.slice/crio-4d1859c0abd187afe80fe0202607f0b90373c09b43359eb09e8b8d0d6166a827 WatchSource:0}: Error finding container 4d1859c0abd187afe80fe0202607f0b90373c09b43359eb09e8b8d0d6166a827: Status 404 returned error can't find the container with id 4d1859c0abd187afe80fe0202607f0b90373c09b43359eb09e8b8d0d6166a827 Jul 28 17:17:05.949825 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:05.949804 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc795a38b_13c3_4b44_ae37_9de0fd450a00.slice/crio-a8338f3ad45ab970c86c3394592213197d1c534914677345fbde425f4d2cc7ae WatchSource:0}: Error finding container a8338f3ad45ab970c86c3394592213197d1c534914677345fbde425f4d2cc7ae: Status 404 returned error can't find the container with id a8338f3ad45ab970c86c3394592213197d1c534914677345fbde425f4d2cc7ae Jul 28 17:17:05.969635 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:05.969610 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b22a64_9b05_4404_be84_b23e2c9d6af6.slice/crio-22398d6cb8103656b355568830cb38dfeeec6413ceadc51d69d4cfc553d02f39 WatchSource:0}: Error finding container 22398d6cb8103656b355568830cb38dfeeec6413ceadc51d69d4cfc553d02f39: Status 404 returned error can't find the container with id 22398d6cb8103656b355568830cb38dfeeec6413ceadc51d69d4cfc553d02f39 Jul 28 17:17:05.970995 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:05.970975 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde1c69f2_7d48_4a1f_9c2f_bf144ec6b80d.slice/crio-6e6536c7425717fad44ca09e5d45ac6aa821f340eac7ad564b05f2a90bfcea79 WatchSource:0}: Error finding container 6e6536c7425717fad44ca09e5d45ac6aa821f340eac7ad564b05f2a90bfcea79: Status 404 returned error can't find the container with id 6e6536c7425717fad44ca09e5d45ac6aa821f340eac7ad564b05f2a90bfcea79 Jul 28 17:17:05.971518 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:05.971485 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc2a32ca_505c_45a6_8ab0_8b994aa479f0.slice/crio-fae29201a749c46787841cd2e26672ddc9f20f65b7db8e281f5fa7561ba6185f WatchSource:0}: Error finding container fae29201a749c46787841cd2e26672ddc9f20f65b7db8e281f5fa7561ba6185f: Status 404 returned error can't find the container with id fae29201a749c46787841cd2e26672ddc9f20f65b7db8e281f5fa7561ba6185f Jul 28 17:17:05.972192 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:05.972166 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09009b39_09ab_475e_bc10_1c0f559e4a07.slice/crio-d39b02157a6c5167ab966e1be4a1771b98589371fbf51884176e08d3e49474cc WatchSource:0}: Error finding container d39b02157a6c5167ab966e1be4a1771b98589371fbf51884176e08d3e49474cc: Status 404 returned error can't find the container with id d39b02157a6c5167ab966e1be4a1771b98589371fbf51884176e08d3e49474cc Jul 28 17:17:05.974038 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:05.973839 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c13cd9b_abcc_40c7_98f1_15d3b843c740.slice/crio-3b5ac50f4961af373e034a54d138a4471d0f820bfeb3dd53f16f29c2c8a6a417 WatchSource:0}: Error finding container 3b5ac50f4961af373e034a54d138a4471d0f820bfeb3dd53f16f29c2c8a6a417: Status 404 returned error can't find the container with id 3b5ac50f4961af373e034a54d138a4471d0f820bfeb3dd53f16f29c2c8a6a417 Jul 28 17:17:05.974525 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:05.974501 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2068e262_1dfc_4587_a4f7_a2bc8f2d2ea5.slice/crio-f97b4c6f9d24033ca82a4b9b588cd626dcd185d154d9b427d5478f0593f58f94 WatchSource:0}: Error finding container f97b4c6f9d24033ca82a4b9b588cd626dcd185d154d9b427d5478f0593f58f94: Status 404 returned error can't find the container with id f97b4c6f9d24033ca82a4b9b588cd626dcd185d154d9b427d5478f0593f58f94 Jul 28 17:17:06.036308 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.036113 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:12:05 +0000 UTC" deadline="2028-03-01 17:21:44.271829749 +0000 UTC" Jul 28 17:17:06.036308 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.036304 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13968h4m38.235528817s" Jul 28 17:17:06.138298 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.138261 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:06.138460 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:06.138392 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:06.149179 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.149153 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" event={"ID":"4746d2d48c5a0af4bf6819c4fa3b9ae9","Type":"ContainerStarted","Data":"a99438705bf9f9547e43bdfaf316ad05f951044bbaf72fc3b96460c82ffd2b8f"} Jul 28 17:17:06.150195 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.150172 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" event={"ID":"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5","Type":"ContainerStarted","Data":"f97b4c6f9d24033ca82a4b9b588cd626dcd185d154d9b427d5478f0593f58f94"} Jul 28 17:17:06.151219 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.151199 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" event={"ID":"0c13cd9b-abcc-40c7-98f1-15d3b843c740","Type":"ContainerStarted","Data":"3b5ac50f4961af373e034a54d138a4471d0f820bfeb3dd53f16f29c2c8a6a417"} Jul 28 17:17:06.152241 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.152217 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xtm4c" event={"ID":"de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d","Type":"ContainerStarted","Data":"6e6536c7425717fad44ca09e5d45ac6aa821f340eac7ad564b05f2a90bfcea79"} Jul 28 17:17:06.153227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.153205 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87p9h" event={"ID":"c795a38b-13c3-4b44-ae37-9de0fd450a00","Type":"ContainerStarted","Data":"a8338f3ad45ab970c86c3394592213197d1c534914677345fbde425f4d2cc7ae"} Jul 28 17:17:06.154196 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.154173 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4vvsd" event={"ID":"09009b39-09ab-475e-bc10-1c0f559e4a07","Type":"ContainerStarted","Data":"d39b02157a6c5167ab966e1be4a1771b98589371fbf51884176e08d3e49474cc"} Jul 28 17:17:06.155137 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.155116 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-flxnq" event={"ID":"bc2a32ca-505c-45a6-8ab0-8b994aa479f0","Type":"ContainerStarted","Data":"fae29201a749c46787841cd2e26672ddc9f20f65b7db8e281f5fa7561ba6185f"} Jul 28 17:17:06.156122 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.156095 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" event={"ID":"54b22a64-9b05-4404-be84-b23e2c9d6af6","Type":"ContainerStarted","Data":"22398d6cb8103656b355568830cb38dfeeec6413ceadc51d69d4cfc553d02f39"} Jul 28 17:17:06.157072 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.157046 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8f82" event={"ID":"ed55c986-c239-4f25-a72f-539854fa246d","Type":"ContainerStarted","Data":"4d1859c0abd187afe80fe0202607f0b90373c09b43359eb09e8b8d0d6166a827"} Jul 28 17:17:06.157956 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.157939 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fbcx8" event={"ID":"12f6882d-5ac2-4ab2-b01f-b295d9eba005","Type":"ContainerStarted","Data":"3f7134622e4a9b7cd85f9f7a3c85fb515c3adfc0aaa487c5feb01a80d484e845"} Jul 28 17:17:06.165334 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.165284 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" podStartSLOduration=2.165265219 podStartE2EDuration="2.165265219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:06.164600258 +0000 UTC m=+3.537148854" watchObservedRunningTime="2026-07-28 17:17:06.165265219 +0000 UTC m=+3.537813816" Jul 28 17:17:06.248937 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.248914 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:17:06.609124 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.609040 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:06.609295 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:06.609228 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:06.609295 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:06.609291 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret podName:6c71f50e-2184-425b-9d24-0e2c2aebcaac nodeName:}" failed. No retries permitted until 2026-07-28 17:17:08.609273604 +0000 UTC m=+5.981822184 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret") pod "global-pull-secret-syncer-tjmr6" (UID: "6c71f50e-2184-425b-9d24-0e2c2aebcaac") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:06.710005 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.709853 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:06.710197 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:06.710180 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:06.710271 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:06.710248 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs podName:e761eac4-98d4-494a-be69-82b2f0e6e4d4 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:08.710229878 +0000 UTC m=+6.082778455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs") pod "network-metrics-daemon-s9qjh" (UID: "e761eac4-98d4-494a-be69-82b2f0e6e4d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:06.811515 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:06.811002 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j66z7\" (UniqueName: \"kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7\") pod \"network-check-target-5g2mj\" (UID: \"4701c1b9-f966-41c6-bd60-86d9fc2bf11e\") " pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:06.811515 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:06.811210 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:06.811515 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:06.811236 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:06.811515 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:06.811249 2576 projected.go:196] Error preparing data for projected volume kube-api-access-j66z7 for pod openshift-network-diagnostics/network-check-target-5g2mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:06.811515 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:06.811309 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7 podName:4701c1b9-f966-41c6-bd60-86d9fc2bf11e nodeName:}" failed. No retries permitted until 2026-07-28 17:17:08.811290016 +0000 UTC m=+6.183838613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j66z7" (UniqueName: "kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7") pod "network-check-target-5g2mj" (UID: "4701c1b9-f966-41c6-bd60-86d9fc2bf11e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:07.037404 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:07.037313 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:12:05 +0000 UTC" deadline="2028-02-18 06:09:55.844083854 +0000 UTC" Jul 28 17:17:07.037404 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:07.037355 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13668h52m48.806733204s" Jul 28 17:17:07.139562 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:07.138901 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:07.139562 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:07.139032 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:07.139562 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:07.139421 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:07.139562 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:07.139518 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:07.169842 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:07.169775 2576 generic.go:356] "Generic (PLEG): container finished" podID="42785eed679f9257ac8637d2df48ef77" containerID="5d2c44fd4aa37f8a22b3e8d332da9f0a24250ca502be03c1adc5b11948169a85" exitCode=0 Jul 28 17:17:07.170034 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:07.169895 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" event={"ID":"42785eed679f9257ac8637d2df48ef77","Type":"ContainerDied","Data":"5d2c44fd4aa37f8a22b3e8d332da9f0a24250ca502be03c1adc5b11948169a85"} Jul 28 17:17:08.138558 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:08.138077 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:08.138558 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:08.138203 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:08.186461 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:08.186425 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" event={"ID":"42785eed679f9257ac8637d2df48ef77","Type":"ContainerStarted","Data":"3be7c35f1d39a4bf829f136dcd852d1b20d126cbf9e90761d006fc15e9d57181"} Jul 28 17:17:08.204103 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:08.204047 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" podStartSLOduration=4.204026535 podStartE2EDuration="4.204026535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:08.202463277 +0000 UTC m=+5.575011876" watchObservedRunningTime="2026-07-28 17:17:08.204026535 +0000 UTC m=+5.576575133" Jul 28 17:17:08.627683 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:08.627645 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:08.627900 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:08.627831 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:08.627976 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:08.627912 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret podName:6c71f50e-2184-425b-9d24-0e2c2aebcaac nodeName:}" failed. No retries permitted until 2026-07-28 17:17:12.627889506 +0000 UTC m=+10.000438095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret") pod "global-pull-secret-syncer-tjmr6" (UID: "6c71f50e-2184-425b-9d24-0e2c2aebcaac") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:08.728594 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:08.728554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:08.728810 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:08.728779 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:08.728888 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:08.728859 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs podName:e761eac4-98d4-494a-be69-82b2f0e6e4d4 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:12.728842241 +0000 UTC m=+10.101390817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs") pod "network-metrics-daemon-s9qjh" (UID: "e761eac4-98d4-494a-be69-82b2f0e6e4d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:08.830051 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:08.829294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j66z7\" (UniqueName: \"kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7\") pod \"network-check-target-5g2mj\" (UID: \"4701c1b9-f966-41c6-bd60-86d9fc2bf11e\") " pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:08.830051 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:08.829529 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:08.830051 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:08.829551 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:08.830051 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:08.829563 2576 projected.go:196] Error preparing data for projected volume kube-api-access-j66z7 for pod openshift-network-diagnostics/network-check-target-5g2mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:08.830051 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:08.829629 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7 podName:4701c1b9-f966-41c6-bd60-86d9fc2bf11e nodeName:}" failed. No retries permitted until 2026-07-28 17:17:12.829610584 +0000 UTC m=+10.202159160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j66z7" (UniqueName: "kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7") pod "network-check-target-5g2mj" (UID: "4701c1b9-f966-41c6-bd60-86d9fc2bf11e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:09.139377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:09.138901 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:09.139377 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:09.139027 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:09.139377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:09.139100 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:09.139377 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:09.139228 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:10.138825 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:10.138776 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:10.139021 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:10.138934 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:11.138968 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:11.138925 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:11.138968 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:11.138971 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:11.139390 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:11.139077 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:11.139570 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:11.139526 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:12.138978 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:12.138383 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:12.138978 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:12.138526 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:12.663169 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:12.663129 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:12.663367 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:12.663293 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:12.663425 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:12.663366 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret podName:6c71f50e-2184-425b-9d24-0e2c2aebcaac nodeName:}" failed. No retries permitted until 2026-07-28 17:17:20.66334594 +0000 UTC m=+18.035894533 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret") pod "global-pull-secret-syncer-tjmr6" (UID: "6c71f50e-2184-425b-9d24-0e2c2aebcaac") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:12.764606 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:12.764015 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:12.764606 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:12.764200 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:12.764606 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:12.764273 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs podName:e761eac4-98d4-494a-be69-82b2f0e6e4d4 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:20.764245224 +0000 UTC m=+18.136793811 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs") pod "network-metrics-daemon-s9qjh" (UID: "e761eac4-98d4-494a-be69-82b2f0e6e4d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:12.864877 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:12.864811 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j66z7\" (UniqueName: \"kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7\") pod \"network-check-target-5g2mj\" (UID: \"4701c1b9-f966-41c6-bd60-86d9fc2bf11e\") " pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:12.865053 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:12.865039 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:12.865110 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:12.865064 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:12.865110 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:12.865076 2576 projected.go:196] Error preparing data for projected volume kube-api-access-j66z7 for pod openshift-network-diagnostics/network-check-target-5g2mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:12.865211 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:12.865135 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7 podName:4701c1b9-f966-41c6-bd60-86d9fc2bf11e nodeName:}" failed. No retries permitted until 2026-07-28 17:17:20.865118508 +0000 UTC m=+18.237667100 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j66z7" (UniqueName: "kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7") pod "network-check-target-5g2mj" (UID: "4701c1b9-f966-41c6-bd60-86d9fc2bf11e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:13.139565 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:13.139474 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:13.140065 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:13.139588 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:13.140065 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:13.139686 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:13.140065 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:13.139777 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:14.138767 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:14.138735 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:14.138936 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:14.138888 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:15.138416 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:15.138384 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:15.138888 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:15.138514 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:15.138888 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:15.138600 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:15.138888 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:15.138700 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:16.138876 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:16.138837 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:16.139321 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:16.138952 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:17.138892 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:17.138808 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:17.139055 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:17.138815 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:17.139055 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:17.138952 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:17.139055 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:17.139025 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:18.138742 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:18.138710 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:18.138925 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:18.138845 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:19.138254 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:19.138216 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:19.138701 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:19.138220 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:19.138701 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:19.138358 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:19.138701 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:19.138439 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:20.138390 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:20.138347 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:20.138886 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:20.138495 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:20.724581 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:20.724543 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:20.724772 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:20.724705 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:20.724772 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:20.724769 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret podName:6c71f50e-2184-425b-9d24-0e2c2aebcaac nodeName:}" failed. No retries permitted until 2026-07-28 17:17:36.724754656 +0000 UTC m=+34.097303230 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret") pod "global-pull-secret-syncer-tjmr6" (UID: "6c71f50e-2184-425b-9d24-0e2c2aebcaac") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:20.825607 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:20.825561 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:20.825823 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:20.825731 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:20.825823 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:20.825810 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs podName:e761eac4-98d4-494a-be69-82b2f0e6e4d4 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:36.825793593 +0000 UTC m=+34.198342180 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs") pod "network-metrics-daemon-s9qjh" (UID: "e761eac4-98d4-494a-be69-82b2f0e6e4d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:20.926479 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:20.926442 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j66z7\" (UniqueName: \"kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7\") pod \"network-check-target-5g2mj\" (UID: \"4701c1b9-f966-41c6-bd60-86d9fc2bf11e\") " pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:20.926645 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:20.926623 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:20.926724 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:20.926654 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:20.926724 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:20.926669 2576 projected.go:196] Error preparing data for projected volume kube-api-access-j66z7 for pod openshift-network-diagnostics/network-check-target-5g2mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:20.926829 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:20.926738 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7 podName:4701c1b9-f966-41c6-bd60-86d9fc2bf11e nodeName:}" failed. No retries permitted until 2026-07-28 17:17:36.926717544 +0000 UTC m=+34.299266142 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j66z7" (UniqueName: "kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7") pod "network-check-target-5g2mj" (UID: "4701c1b9-f966-41c6-bd60-86d9fc2bf11e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:21.138302 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:21.138264 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:21.138485 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:21.138408 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:21.138485 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:21.138462 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:21.138902 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:21.138609 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:22.138406 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:22.138369 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:22.138589 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:22.138507 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:23.139083 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:23.139049 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:23.139515 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:23.139161 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:23.139515 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:23.139255 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:23.139515 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:23.139383 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:24.138648 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:24.138615 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:24.138810 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:24.138728 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:25.138150 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.137853 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:25.138772 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.137890 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:25.138772 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:25.138235 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:25.138772 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:25.138355 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:25.226949 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.226841 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" event={"ID":"54b22a64-9b05-4404-be84-b23e2c9d6af6","Type":"ContainerStarted","Data":"84989f58770cf65b72c2fc25a566beb8e164050624d284213b018a824b5ce50d"} Jul 28 17:17:25.227134 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.227111 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" event={"ID":"54b22a64-9b05-4404-be84-b23e2c9d6af6","Type":"ContainerStarted","Data":"620c9680db1778d68927d04ef01cbe5b65a3bc66b4d5a5c109d1372e763c4e65"} Jul 28 17:17:25.227267 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.227248 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" event={"ID":"54b22a64-9b05-4404-be84-b23e2c9d6af6","Type":"ContainerStarted","Data":"ca42c87257284f67ec1498e367ee7751fd1b3202edd65e1eaf7877665e767acf"} Jul 28 17:17:25.227363 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.227348 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" event={"ID":"54b22a64-9b05-4404-be84-b23e2c9d6af6","Type":"ContainerStarted","Data":"30fb09e1f62e098051bdc70fdbe8dc0b4503e9cc30ede53ed2fbcc1caba33ae1"} Jul 28 17:17:25.227479 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.227467 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" event={"ID":"54b22a64-9b05-4404-be84-b23e2c9d6af6","Type":"ContainerStarted","Data":"be19f2216eaa8c544ec87d80bdc1dc7d4b48478b00213870fb1e668a7d1d97d9"} Jul 28 17:17:25.227559 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.227545 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" event={"ID":"54b22a64-9b05-4404-be84-b23e2c9d6af6","Type":"ContainerStarted","Data":"81ac1ded2ba54298e95af0c8e3702a37c10d56545183bc177091b94b003847bd"} Jul 28 17:17:25.229422 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.229388 2576 generic.go:356] "Generic (PLEG): container finished" podID="ed55c986-c239-4f25-a72f-539854fa246d" containerID="5010e053acd8215f4255254cb9de05b67cd51d0a369914ba5f20818d0604d036" exitCode=0 Jul 28 17:17:25.229555 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.229519 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8f82" event={"ID":"ed55c986-c239-4f25-a72f-539854fa246d","Type":"ContainerDied","Data":"5010e053acd8215f4255254cb9de05b67cd51d0a369914ba5f20818d0604d036"} Jul 28 17:17:25.232492 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.232470 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" event={"ID":"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5","Type":"ContainerStarted","Data":"a3107624105acbfa48b0dc255103d46b9fe0f9ad7e706db1ec29c4536ef71c39"} Jul 28 17:17:25.234067 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.234048 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" event={"ID":"0c13cd9b-abcc-40c7-98f1-15d3b843c740","Type":"ContainerStarted","Data":"fe98a2c4b0e3213fa3fdee360d084c2ddbe036c88ad7f1d2108ebdc9df03d760"} Jul 28 17:17:25.236000 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.235724 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xtm4c" event={"ID":"de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d","Type":"ContainerStarted","Data":"8ae179bf0ed3b13b24dea3839e82ff56f4fe3895f4b2a0257c6390528012e942"} Jul 28 17:17:25.237454 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.237432 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87p9h" event={"ID":"c795a38b-13c3-4b44-ae37-9de0fd450a00","Type":"ContainerStarted","Data":"658d313a24c36289235f2a30814db66fb6ded736572b7f30403d4bf36b00c130"} Jul 28 17:17:25.239034 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.239004 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4vvsd" event={"ID":"09009b39-09ab-475e-bc10-1c0f559e4a07","Type":"ContainerStarted","Data":"3e5ef5511c1f9d0ab6e1cb864d7d7faa9e26d29717be20141c3cd6aa83db9913"} Jul 28 17:17:25.240566 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.240546 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-flxnq" event={"ID":"bc2a32ca-505c-45a6-8ab0-8b994aa479f0","Type":"ContainerStarted","Data":"5f5e982b424b82a3dcc69261f47858b4347688f3e84ea5f5581aefc8a5ceaf05"} Jul 28 17:17:25.269793 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.269742 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-87p9h" podStartSLOduration=3.889147711 podStartE2EDuration="22.269726024s" totalImagesPullingTime="18.380578313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:25.269412188 +0000 UTC m=+22.641960784" watchObservedRunningTime="2026-07-28 17:17:25.269726024 +0000 UTC m=+22.642274621" Jul 28 17:17:25.289331 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.289275 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xtm4c" podStartSLOduration=11.710050821 podStartE2EDuration="22.289260059s" totalImagesPullingTime="10.579209238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:25.288437406 +0000 UTC m=+22.660986001" watchObservedRunningTime="2026-07-28 17:17:25.289260059 +0000 UTC m=+22.661808659" Jul 28 17:17:25.304832 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.304752 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4vvsd" podStartSLOduration=3.933450148 podStartE2EDuration="22.304738253s" totalImagesPullingTime="18.371288105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:25.304539555 +0000 UTC m=+22.677088151" watchObservedRunningTime="2026-07-28 17:17:25.304738253 +0000 UTC m=+22.677286849" Jul 28 17:17:25.321655 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.321616 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-flxnq" podStartSLOduration=3.8995228600000003 podStartE2EDuration="22.321603827s" totalImagesPullingTime="18.422080967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:25.321428894 +0000 UTC m=+22.693977488" watchObservedRunningTime="2026-07-28 17:17:25.321603827 +0000 UTC m=+22.694152423" Jul 28 17:17:25.339384 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:25.339338 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w5dwp" podStartSLOduration=3.964135096 podStartE2EDuration="22.339324456s" totalImagesPullingTime="18.37518936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:25.338676732 +0000 UTC m=+22.711225325" watchObservedRunningTime="2026-07-28 17:17:25.339324456 +0000 UTC m=+22.711873051" Jul 28 17:17:26.012102 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:26.012078 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 17:17:26.046126 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:26.045971 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:17:26.012095001Z","UUID":"311e9bc9-8b81-42de-9260-7dde69f1e4f4","Handler":null,"Name":"","Endpoint":""} Jul 28 17:17:26.047451 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:26.047435 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 17:17:26.047551 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:26.047468 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 17:17:26.138471 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:26.138440 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:26.138961 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:26.138570 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:26.245378 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:26.245292 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" event={"ID":"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5","Type":"ContainerStarted","Data":"a740ec05ece673ff677f59ebe9e4fee075d78634984bd196791f972eff0b8c46"} Jul 28 17:17:26.246748 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:26.246709 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fbcx8" event={"ID":"12f6882d-5ac2-4ab2-b01f-b295d9eba005","Type":"ContainerStarted","Data":"b7eaaae0560d0c0bd80d633f3af2bb1f65195ff226a765a7607e8a33f51daa45"} Jul 28 17:17:26.263910 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:26.263864 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fbcx8" podStartSLOduration=4.861397778 podStartE2EDuration="23.2638479s" totalImagesPullingTime="18.402450122s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:26.26356667 +0000 UTC m=+23.636115266" watchObservedRunningTime="2026-07-28 17:17:26.2638479 +0000 UTC m=+23.636396495" Jul 28 17:17:27.138905 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:27.138860 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:27.139459 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:27.138860 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:27.139459 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:27.138995 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:27.139459 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:27.139024 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:27.255837 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:27.255778 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" event={"ID":"54b22a64-9b05-4404-be84-b23e2c9d6af6","Type":"ContainerStarted","Data":"a491d8f7465efb2401e59dedc5d647c71f596b9913f6f87238159ce7d64590fe"} Jul 28 17:17:28.138315 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:28.138133 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:28.138479 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:28.138389 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:28.259839 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:28.259756 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" event={"ID":"2068e262-1dfc-4587-a4f7-a2bc8f2d2ea5","Type":"ContainerStarted","Data":"fc0c669e03522d30024008a9ac9b2ce74d362c6f0259a101da313f96223b85a8"} Jul 28 17:17:28.279358 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:28.279312 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvwzr" podStartSLOduration=3.63195208 podStartE2EDuration="25.279296564s" totalImagesPullingTime="21.647344484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:28.278040188 +0000 UTC m=+25.650588774" watchObservedRunningTime="2026-07-28 17:17:28.279296564 +0000 UTC m=+25.651845161" Jul 28 17:17:29.138153 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:29.138118 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:29.138330 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:29.138118 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:29.138330 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:29.138258 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:29.138330 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:29.138305 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:29.509200 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:29.509110 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:29.509815 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:29.509776 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:30.138797 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:30.138761 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:30.138911 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:30.138877 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:30.268763 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:30.268727 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" event={"ID":"54b22a64-9b05-4404-be84-b23e2c9d6af6","Type":"ContainerStarted","Data":"87b57e1b6efc76badb64f11206e659bafd03288b52e0fffdc57193a827824fca"} Jul 28 17:17:30.269056 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:30.269035 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:30.269685 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:30.269666 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xtm4c" Jul 28 17:17:30.298578 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:30.298364 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" podStartSLOduration=8.844065001 podStartE2EDuration="27.298349206s" totalImagesPullingTime="18.454284205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:30.297846016 +0000 UTC m=+27.670394613" watchObservedRunningTime="2026-07-28 17:17:30.298349206 +0000 UTC m=+27.670897801" Jul 28 17:17:31.138022 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:31.137986 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:31.138681 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:31.137989 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:31.138681 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:31.138094 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:31.138681 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:31.138170 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:31.272712 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:31.272672 2576 generic.go:356] "Generic (PLEG): container finished" podID="ed55c986-c239-4f25-a72f-539854fa246d" containerID="d3abc2fa0e9813304aea33e0f54655763a826ed78a11ac9f08aac47f6ff8d22b" exitCode=0 Jul 28 17:17:31.272874 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:31.272731 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8f82" event={"ID":"ed55c986-c239-4f25-a72f-539854fa246d","Type":"ContainerDied","Data":"d3abc2fa0e9813304aea33e0f54655763a826ed78a11ac9f08aac47f6ff8d22b"} Jul 28 17:17:31.273223 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:31.273192 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:31.273296 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:31.273258 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:31.273449 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:31.273305 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:31.288398 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:31.288366 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:31.288547 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:31.288534 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:17:32.138692 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:32.138609 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:32.139167 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:32.138738 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:32.416926 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:32.416845 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tjmr6"] Jul 28 17:17:32.417081 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:32.417072 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:32.417296 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:32.417238 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:32.417633 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:32.417608 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5g2mj"] Jul 28 17:17:32.417731 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:32.417709 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:32.417845 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:32.417824 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:32.420674 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:32.420645 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s9qjh"] Jul 28 17:17:32.420816 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:32.420761 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:32.421299 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:32.420884 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:33.280511 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:33.280231 2576 generic.go:356] "Generic (PLEG): container finished" podID="ed55c986-c239-4f25-a72f-539854fa246d" containerID="3a9498b68817b464425fcdfa51fa76f883cc8019476c69ad9fccec88ffe491de" exitCode=0 Jul 28 17:17:33.280970 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:33.280308 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8f82" event={"ID":"ed55c986-c239-4f25-a72f-539854fa246d","Type":"ContainerDied","Data":"3a9498b68817b464425fcdfa51fa76f883cc8019476c69ad9fccec88ffe491de"} Jul 28 17:17:34.138025 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:34.137928 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:34.138185 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:34.138049 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:34.138185 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:34.138059 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:34.138185 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:34.138086 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:34.138334 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:34.138181 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:34.138334 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:34.138236 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:35.286911 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:35.286873 2576 generic.go:356] "Generic (PLEG): container finished" podID="ed55c986-c239-4f25-a72f-539854fa246d" containerID="86ddba2d5171227919189894fe3b9ed1ae62ecb8edcb96a6bd15ab0aef8ff896" exitCode=0 Jul 28 17:17:35.287449 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:35.286921 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8f82" event={"ID":"ed55c986-c239-4f25-a72f-539854fa246d","Type":"ContainerDied","Data":"86ddba2d5171227919189894fe3b9ed1ae62ecb8edcb96a6bd15ab0aef8ff896"} Jul 28 17:17:36.138757 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:36.138720 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:36.138757 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:36.138751 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:36.139011 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:36.138751 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:36.139011 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:36.138865 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmr6" podUID="6c71f50e-2184-425b-9d24-0e2c2aebcaac" Jul 28 17:17:36.139011 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:36.138991 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s9qjh" podUID="e761eac4-98d4-494a-be69-82b2f0e6e4d4" Jul 28 17:17:36.139137 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:36.139080 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2mj" podUID="4701c1b9-f966-41c6-bd60-86d9fc2bf11e" Jul 28 17:17:36.748493 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:36.748462 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:36.749078 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:36.748605 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:36.749078 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:36.748662 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret podName:6c71f50e-2184-425b-9d24-0e2c2aebcaac nodeName:}" failed. No retries permitted until 2026-07-28 17:18:08.748646281 +0000 UTC m=+66.121194877 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret") pod "global-pull-secret-syncer-tjmr6" (UID: "6c71f50e-2184-425b-9d24-0e2c2aebcaac") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:36.849666 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:36.849631 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:36.849843 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:36.849820 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:36.849923 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:36.849901 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs podName:e761eac4-98d4-494a-be69-82b2f0e6e4d4 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:08.849881823 +0000 UTC m=+66.222430412 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs") pod "network-metrics-daemon-s9qjh" (UID: "e761eac4-98d4-494a-be69-82b2f0e6e4d4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:36.944288 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:36.944240 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeReady" Jul 28 17:17:36.944447 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:36.944417 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 17:17:36.950010 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:36.949978 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j66z7\" (UniqueName: \"kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7\") pod \"network-check-target-5g2mj\" (UID: \"4701c1b9-f966-41c6-bd60-86d9fc2bf11e\") " pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:36.950144 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:36.950106 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:36.950144 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:36.950128 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:36.950144 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:36.950143 2576 projected.go:196] Error preparing data for projected volume kube-api-access-j66z7 for pod openshift-network-diagnostics/network-check-target-5g2mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:36.950275 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:36.950199 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7 podName:4701c1b9-f966-41c6-bd60-86d9fc2bf11e nodeName:}" failed. No retries permitted until 2026-07-28 17:18:08.950182416 +0000 UTC m=+66.322731010 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-j66z7" (UniqueName: "kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7") pod "network-check-target-5g2mj" (UID: "4701c1b9-f966-41c6-bd60-86d9fc2bf11e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:36.995159 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:36.995122 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tx79n"] Jul 28 17:17:36.997302 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:36.997277 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.000040 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:36.999973 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qjb4d\"" Jul 28 17:17:37.000220 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.000203 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 17:17:37.000460 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.000438 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 17:17:37.000726 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.000700 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xg2v2"] Jul 28 17:17:37.002368 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.002348 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xg2v2" Jul 28 17:17:37.004764 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.004739 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 17:17:37.004888 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.004761 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 17:17:37.004888 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.004801 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 17:17:37.004888 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.004810 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wqh9j\"" Jul 28 17:17:37.010315 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.010020 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tx79n"] Jul 28 17:17:37.012399 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.012368 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xg2v2"] Jul 28 17:17:37.151432 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.151395 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a20f6b5-d596-481d-8de8-edb57100e326-config-volume\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.151612 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.151445 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdmz8\" (UniqueName: \"kubernetes.io/projected/db7b2d75-c2fa-4f26-986e-ce1164ae5b67-kube-api-access-jdmz8\") pod \"ingress-canary-xg2v2\" (UID: \"db7b2d75-c2fa-4f26-986e-ce1164ae5b67\") " pod="openshift-ingress-canary/ingress-canary-xg2v2" Jul 28 17:17:37.151612 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.151525 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffbzw\" (UniqueName: \"kubernetes.io/projected/0a20f6b5-d596-481d-8de8-edb57100e326-kube-api-access-ffbzw\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.151612 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.151587 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db7b2d75-c2fa-4f26-986e-ce1164ae5b67-cert\") pod \"ingress-canary-xg2v2\" (UID: \"db7b2d75-c2fa-4f26-986e-ce1164ae5b67\") " pod="openshift-ingress-canary/ingress-canary-xg2v2" Jul 28 17:17:37.151725 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.151616 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a20f6b5-d596-481d-8de8-edb57100e326-tmp-dir\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.151725 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.151658 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.252388 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.252306 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db7b2d75-c2fa-4f26-986e-ce1164ae5b67-cert\") pod \"ingress-canary-xg2v2\" (UID: \"db7b2d75-c2fa-4f26-986e-ce1164ae5b67\") " pod="openshift-ingress-canary/ingress-canary-xg2v2" Jul 28 17:17:37.252388 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.252347 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a20f6b5-d596-481d-8de8-edb57100e326-tmp-dir\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.252388 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.252366 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.252648 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:37.252483 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:17:37.252648 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.252501 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a20f6b5-d596-481d-8de8-edb57100e326-config-volume\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.252648 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:37.252538 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls podName:0a20f6b5-d596-481d-8de8-edb57100e326 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:37.752523669 +0000 UTC m=+35.125072243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls") pod "dns-default-tx79n" (UID: "0a20f6b5-d596-481d-8de8-edb57100e326") : secret "dns-default-metrics-tls" not found Jul 28 17:17:37.252648 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.252565 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdmz8\" (UniqueName: \"kubernetes.io/projected/db7b2d75-c2fa-4f26-986e-ce1164ae5b67-kube-api-access-jdmz8\") pod \"ingress-canary-xg2v2\" (UID: \"db7b2d75-c2fa-4f26-986e-ce1164ae5b67\") " pod="openshift-ingress-canary/ingress-canary-xg2v2" Jul 28 17:17:37.252648 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.252620 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffbzw\" (UniqueName: \"kubernetes.io/projected/0a20f6b5-d596-481d-8de8-edb57100e326-kube-api-access-ffbzw\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.253047 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.253024 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a20f6b5-d596-481d-8de8-edb57100e326-tmp-dir\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.253209 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.253191 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a20f6b5-d596-481d-8de8-edb57100e326-config-volume\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.257435 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.257413 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db7b2d75-c2fa-4f26-986e-ce1164ae5b67-cert\") pod \"ingress-canary-xg2v2\" (UID: \"db7b2d75-c2fa-4f26-986e-ce1164ae5b67\") " pod="openshift-ingress-canary/ingress-canary-xg2v2" Jul 28 17:17:37.262882 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.262857 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffbzw\" (UniqueName: \"kubernetes.io/projected/0a20f6b5-d596-481d-8de8-edb57100e326-kube-api-access-ffbzw\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.263007 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.262961 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdmz8\" (UniqueName: \"kubernetes.io/projected/db7b2d75-c2fa-4f26-986e-ce1164ae5b67-kube-api-access-jdmz8\") pod \"ingress-canary-xg2v2\" (UID: \"db7b2d75-c2fa-4f26-986e-ce1164ae5b67\") " pod="openshift-ingress-canary/ingress-canary-xg2v2" Jul 28 17:17:37.318026 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.317987 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xg2v2" Jul 28 17:17:37.470143 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.470111 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xg2v2"] Jul 28 17:17:37.474033 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:37.474000 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb7b2d75_c2fa_4f26_986e_ce1164ae5b67.slice/crio-37258b2c9e0e78e25605b2e7c72d8f7e9ae30c945a11e762456e6dcf14753757 WatchSource:0}: Error finding container 37258b2c9e0e78e25605b2e7c72d8f7e9ae30c945a11e762456e6dcf14753757: Status 404 returned error can't find the container with id 37258b2c9e0e78e25605b2e7c72d8f7e9ae30c945a11e762456e6dcf14753757 Jul 28 17:17:37.756575 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:37.756542 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:37.757191 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:37.756751 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:17:37.757191 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:37.756841 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls podName:0a20f6b5-d596-481d-8de8-edb57100e326 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:38.756824899 +0000 UTC m=+36.129373473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls") pod "dns-default-tx79n" (UID: "0a20f6b5-d596-481d-8de8-edb57100e326") : secret "dns-default-metrics-tls" not found Jul 28 17:17:38.138670 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:38.138613 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:17:38.138670 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:38.138659 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:17:38.138914 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:38.138631 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:17:38.141803 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:38.141713 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:17:38.141803 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:38.141733 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 17:17:38.141803 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:38.141767 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 17:17:38.141803 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:38.141722 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:17:38.142075 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:38.141797 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvwzr\"" Jul 28 17:17:38.142854 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:38.142834 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qzphg\"" Jul 28 17:17:38.295947 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:38.295905 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xg2v2" event={"ID":"db7b2d75-c2fa-4f26-986e-ce1164ae5b67","Type":"ContainerStarted","Data":"37258b2c9e0e78e25605b2e7c72d8f7e9ae30c945a11e762456e6dcf14753757"} Jul 28 17:17:38.765574 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:38.765539 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:38.766121 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:38.765708 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:17:38.766121 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:38.765808 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls podName:0a20f6b5-d596-481d-8de8-edb57100e326 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:40.765773147 +0000 UTC m=+38.138321737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls") pod "dns-default-tx79n" (UID: "0a20f6b5-d596-481d-8de8-edb57100e326") : secret "dns-default-metrics-tls" not found Jul 28 17:17:40.303798 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:40.303750 2576 generic.go:356] "Generic (PLEG): container finished" podID="ed55c986-c239-4f25-a72f-539854fa246d" containerID="7bc843d561323900a6f5e955badf8893794a4352e8251f9cd7f8109fec371da7" exitCode=0 Jul 28 17:17:40.304210 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:40.303823 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8f82" event={"ID":"ed55c986-c239-4f25-a72f-539854fa246d","Type":"ContainerDied","Data":"7bc843d561323900a6f5e955badf8893794a4352e8251f9cd7f8109fec371da7"} Jul 28 17:17:40.783504 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:40.783280 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:40.783680 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:40.783456 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:17:40.783680 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:40.783654 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls podName:0a20f6b5-d596-481d-8de8-edb57100e326 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:44.783626111 +0000 UTC m=+42.156174702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls") pod "dns-default-tx79n" (UID: "0a20f6b5-d596-481d-8de8-edb57100e326") : secret "dns-default-metrics-tls" not found Jul 28 17:17:41.309394 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:41.309358 2576 generic.go:356] "Generic (PLEG): container finished" podID="ed55c986-c239-4f25-a72f-539854fa246d" containerID="53dbca028b84e0855eda49b51ae3768e9f9aeefb4ca5cf9190e79bb6434a0e9d" exitCode=0 Jul 28 17:17:41.309874 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:41.309416 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8f82" event={"ID":"ed55c986-c239-4f25-a72f-539854fa246d","Type":"ContainerDied","Data":"53dbca028b84e0855eda49b51ae3768e9f9aeefb4ca5cf9190e79bb6434a0e9d"} Jul 28 17:17:42.315201 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:42.315162 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8f82" event={"ID":"ed55c986-c239-4f25-a72f-539854fa246d","Type":"ContainerStarted","Data":"19922c12f9ea1f9a5cc0f35ff57c0aa489773e24c2ae70555b57eff61fce1ecf"} Jul 28 17:17:42.316452 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:42.316428 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xg2v2" event={"ID":"db7b2d75-c2fa-4f26-986e-ce1164ae5b67","Type":"ContainerStarted","Data":"7f56450f226462c6cfd1419e25f051627afe9133398d47826217a63fc6d05192"} Jul 28 17:17:42.339669 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:42.339624 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-l8f82" podStartSLOduration=8.969674315 podStartE2EDuration="39.339608818s" totalImagesPullingTime="30.369934503s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:42.33849471 +0000 UTC m=+39.711043305" watchObservedRunningTime="2026-07-28 17:17:42.339608818 +0000 UTC m=+39.712157414" Jul 28 17:17:42.353892 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:42.353819 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xg2v2" podStartSLOduration=2.406357607 podStartE2EDuration="6.353802371s" totalImagesPullingTime="3.947444764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:42.353087408 +0000 UTC m=+39.725636005" watchObservedRunningTime="2026-07-28 17:17:42.353802371 +0000 UTC m=+39.726350959" Jul 28 17:17:44.810131 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:44.810086 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:44.810549 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:44.810233 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:17:44.810549 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:44.810304 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls podName:0a20f6b5-d596-481d-8de8-edb57100e326 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:52.810285423 +0000 UTC m=+50.182833997 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls") pod "dns-default-tx79n" (UID: "0a20f6b5-d596-481d-8de8-edb57100e326") : secret "dns-default-metrics-tls" not found Jul 28 17:17:51.367872 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.367837 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm"] Jul 28 17:17:51.419683 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.419643 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-fdb6z"] Jul 28 17:17:51.419870 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.419776 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:51.423639 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.423605 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 28 17:17:51.423639 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.423617 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bc2pz\"" Jul 28 17:17:51.423639 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.423636 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 28 17:17:51.423877 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.423660 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 17:17:51.423877 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.423711 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 17:17:51.437403 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.437383 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm"] Jul 28 17:17:51.437499 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.437412 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-fdb6z"] Jul 28 17:17:51.437499 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.437485 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.439665 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.439647 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 28 17:17:51.439798 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.439697 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 28 17:17:51.439798 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.439650 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 17:17:51.440012 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.439994 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vp4z2\"" Jul 28 17:17:51.440074 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.440026 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 17:17:51.444766 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.444748 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 28 17:17:51.551888 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.551852 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgtbd\" (UniqueName: \"kubernetes.io/projected/eef7416b-d05b-4b67-b553-dc593460efb1-kube-api-access-pgtbd\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.551888 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.551890 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5d7e904f-06b5-4dfb-ac37-a97b38ace547-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:51.552084 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.551913 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eef7416b-d05b-4b67-b553-dc593460efb1-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.552084 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.551977 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eef7416b-d05b-4b67-b553-dc593460efb1-service-ca-bundle\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.552084 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.552048 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:51.552179 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.552082 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eef7416b-d05b-4b67-b553-dc593460efb1-serving-cert\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.552179 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.552150 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg85q\" (UniqueName: \"kubernetes.io/projected/5d7e904f-06b5-4dfb-ac37-a97b38ace547-kube-api-access-kg85q\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:51.552239 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.552200 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eef7416b-d05b-4b67-b553-dc593460efb1-tmp\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.552239 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.552223 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/eef7416b-d05b-4b67-b553-dc593460efb1-snapshots\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.653246 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.653164 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eef7416b-d05b-4b67-b553-dc593460efb1-tmp\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.653246 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.653204 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/eef7416b-d05b-4b67-b553-dc593460efb1-snapshots\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.653246 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.653234 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgtbd\" (UniqueName: \"kubernetes.io/projected/eef7416b-d05b-4b67-b553-dc593460efb1-kube-api-access-pgtbd\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.653505 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.653250 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5d7e904f-06b5-4dfb-ac37-a97b38ace547-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:51.653505 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.653269 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eef7416b-d05b-4b67-b553-dc593460efb1-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.653505 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.653295 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eef7416b-d05b-4b67-b553-dc593460efb1-service-ca-bundle\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.653505 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.653318 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:51.653505 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.653337 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eef7416b-d05b-4b67-b553-dc593460efb1-serving-cert\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.653505 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.653369 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg85q\" (UniqueName: \"kubernetes.io/projected/5d7e904f-06b5-4dfb-ac37-a97b38ace547-kube-api-access-kg85q\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:51.653505 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:51.653487 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:17:51.653860 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:51.653563 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls podName:5d7e904f-06b5-4dfb-ac37-a97b38ace547 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:52.153541731 +0000 UTC m=+49.526090323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sfqvm" (UID: "5d7e904f-06b5-4dfb-ac37-a97b38ace547") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:17:51.653860 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.653678 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eef7416b-d05b-4b67-b553-dc593460efb1-tmp\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.653996 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.653977 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eef7416b-d05b-4b67-b553-dc593460efb1-service-ca-bundle\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.654031 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.654013 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5d7e904f-06b5-4dfb-ac37-a97b38ace547-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:51.654064 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.654022 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/eef7416b-d05b-4b67-b553-dc593460efb1-snapshots\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.654457 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.654440 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eef7416b-d05b-4b67-b553-dc593460efb1-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.657272 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.657255 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eef7416b-d05b-4b67-b553-dc593460efb1-serving-cert\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.666391 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.666360 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg85q\" (UniqueName: \"kubernetes.io/projected/5d7e904f-06b5-4dfb-ac37-a97b38ace547-kube-api-access-kg85q\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:51.667212 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.667192 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgtbd\" (UniqueName: \"kubernetes.io/projected/eef7416b-d05b-4b67-b553-dc593460efb1-kube-api-access-pgtbd\") pod \"insights-operator-5885fdb759-fdb6z\" (UID: \"eef7416b-d05b-4b67-b553-dc593460efb1\") " pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.744913 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.744891 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-fdb6z" Jul 28 17:17:51.913377 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:51.913288 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-fdb6z"] Jul 28 17:17:51.916868 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:51.916841 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef7416b_d05b_4b67_b553_dc593460efb1.slice/crio-62f5ee887c3aaa62dafff3de1f0260e076dc92cf07661108a7215717c5bc993b WatchSource:0}: Error finding container 62f5ee887c3aaa62dafff3de1f0260e076dc92cf07661108a7215717c5bc993b: Status 404 returned error can't find the container with id 62f5ee887c3aaa62dafff3de1f0260e076dc92cf07661108a7215717c5bc993b Jul 28 17:17:52.157854 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:52.157816 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:52.158027 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:52.157980 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:17:52.158071 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:52.158047 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls podName:5d7e904f-06b5-4dfb-ac37-a97b38ace547 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:53.158032152 +0000 UTC m=+50.530580726 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sfqvm" (UID: "5d7e904f-06b5-4dfb-ac37-a97b38ace547") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:17:52.341486 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:52.341451 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fdb6z" event={"ID":"eef7416b-d05b-4b67-b553-dc593460efb1","Type":"ContainerStarted","Data":"62f5ee887c3aaa62dafff3de1f0260e076dc92cf07661108a7215717c5bc993b"} Jul 28 17:17:52.863696 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:52.863656 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:17:52.864078 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:52.863834 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:17:52.864078 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:52.863898 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls podName:0a20f6b5-d596-481d-8de8-edb57100e326 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:08.863882706 +0000 UTC m=+66.236431282 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls") pod "dns-default-tx79n" (UID: "0a20f6b5-d596-481d-8de8-edb57100e326") : secret "dns-default-metrics-tls" not found Jul 28 17:17:53.167069 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:53.167000 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:53.167200 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:53.167110 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:17:53.167200 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:53.167162 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls podName:5d7e904f-06b5-4dfb-ac37-a97b38ace547 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:55.167149487 +0000 UTC m=+52.539698061 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sfqvm" (UID: "5d7e904f-06b5-4dfb-ac37-a97b38ace547") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:17:54.347567 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.347534 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fdb6z" event={"ID":"eef7416b-d05b-4b67-b553-dc593460efb1","Type":"ContainerStarted","Data":"ef25cfd46f19147a8b6b6caec06a9326eefc72a8d3b1cc2af6e05e79cd68c427"} Jul 28 17:17:54.368988 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.368933 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-fdb6z" podStartSLOduration=1.337984713 podStartE2EDuration="3.368913962s" totalImagesPullingTime="2.030929249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:54.368445066 +0000 UTC m=+51.740993658" watchObservedRunningTime="2026-07-28 17:17:54.368913962 +0000 UTC m=+51.741462559" Jul 28 17:17:54.707180 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.707093 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9cghg"] Jul 28 17:17:54.723011 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.722983 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9cghg"] Jul 28 17:17:54.723158 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.723083 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.725607 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.725581 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 17:17:54.725733 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.725688 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 17:17:54.725930 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.725915 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mc5b9\"" Jul 28 17:17:54.881833 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.881775 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.881988 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.881836 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eae07057-b8a0-4138-88fd-0ba713b04676-crio-socket\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.881988 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.881903 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn6jt\" (UniqueName: \"kubernetes.io/projected/eae07057-b8a0-4138-88fd-0ba713b04676-kube-api-access-xn6jt\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.881988 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.881972 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eae07057-b8a0-4138-88fd-0ba713b04676-data-volume\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.882100 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.881997 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.982614 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.982551 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.982614 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.982591 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eae07057-b8a0-4138-88fd-0ba713b04676-crio-socket\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.982769 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.982613 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn6jt\" (UniqueName: \"kubernetes.io/projected/eae07057-b8a0-4138-88fd-0ba713b04676-kube-api-access-xn6jt\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.982769 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.982657 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eae07057-b8a0-4138-88fd-0ba713b04676-data-volume\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.982769 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.982681 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.982769 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.982691 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eae07057-b8a0-4138-88fd-0ba713b04676-crio-socket\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.982769 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:54.982701 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:17:54.982769 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:54.982758 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:17:54.983020 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:54.982802 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls podName:eae07057-b8a0-4138-88fd-0ba713b04676 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:55.482762118 +0000 UTC m=+52.855310692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9cghg" (UID: "eae07057-b8a0-4138-88fd-0ba713b04676") : secret "insights-runtime-extractor-tls" not found Jul 28 17:17:54.983020 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:54.982823 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm podName:eae07057-b8a0-4138-88fd-0ba713b04676 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:55.482811161 +0000 UTC m=+52.855359736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9cghg" (UID: "eae07057-b8a0-4138-88fd-0ba713b04676") : configmap "kube-rbac-proxy" not found Jul 28 17:17:54.983020 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.982974 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eae07057-b8a0-4138-88fd-0ba713b04676-data-volume\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:54.991188 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:54.991166 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn6jt\" (UniqueName: \"kubernetes.io/projected/eae07057-b8a0-4138-88fd-0ba713b04676-kube-api-access-xn6jt\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:55.183780 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.183749 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:55.183942 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:55.183893 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:17:55.183978 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:55.183952 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls podName:5d7e904f-06b5-4dfb-ac37-a97b38ace547 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:59.18393404 +0000 UTC m=+56.556482634 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sfqvm" (UID: "5d7e904f-06b5-4dfb-ac37-a97b38ace547") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:17:55.365107 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.365076 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs"] Jul 28 17:17:55.378390 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.378361 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs"] Jul 28 17:17:55.378516 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.378470 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:17:55.380939 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.380913 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 28 17:17:55.380939 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.380915 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:17:55.381149 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.380973 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zwcz8\"" Jul 28 17:17:55.381149 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.380980 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 28 17:17:55.465848 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.465820 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-5msdh"] Jul 28 17:17:55.486103 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.486073 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:17:55.486241 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.486119 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:55.486241 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:55.486214 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:17:55.486338 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:55.486261 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm podName:eae07057-b8a0-4138-88fd-0ba713b04676 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:56.486246503 +0000 UTC m=+53.858795081 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9cghg" (UID: "eae07057-b8a0-4138-88fd-0ba713b04676") : configmap "kube-rbac-proxy" not found Jul 28 17:17:55.486338 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.486278 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-5msdh" Jul 28 17:17:55.486338 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.486301 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:55.486338 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.486308 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-5msdh"] Jul 28 17:17:55.486535 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.486338 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtnwc\" (UniqueName: \"kubernetes.io/projected/b98f002a-2718-48c7-8a9a-8905c76434e9-kube-api-access-dtnwc\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:17:55.487116 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:55.487096 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:17:55.487341 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:55.487327 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls podName:eae07057-b8a0-4138-88fd-0ba713b04676 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:56.487304874 +0000 UTC m=+53.859853465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9cghg" (UID: "eae07057-b8a0-4138-88fd-0ba713b04676") : secret "insights-runtime-extractor-tls" not found Jul 28 17:17:55.490820 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.490775 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-blsdj\"" Jul 28 17:17:55.490943 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.490865 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:17:55.490943 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.490895 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 28 17:17:55.587765 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.587731 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:17:55.587961 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.587845 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl9l9\" (UniqueName: \"kubernetes.io/projected/ffa9c82e-220f-4b99-bf40-4136da9bd925-kube-api-access-hl9l9\") pod \"volume-data-source-validator-549c6d54cf-5msdh\" (UID: \"ffa9c82e-220f-4b99-bf40-4136da9bd925\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-5msdh" Jul 28 17:17:55.587961 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:55.587900 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:17:55.588073 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:55.587968 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls podName:b98f002a-2718-48c7-8a9a-8905c76434e9 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:56.087954562 +0000 UTC m=+53.460503138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls") pod "cluster-samples-operator-6748489656-srchs" (UID: "b98f002a-2718-48c7-8a9a-8905c76434e9") : secret "samples-operator-tls" not found Jul 28 17:17:55.588073 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.587901 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtnwc\" (UniqueName: \"kubernetes.io/projected/b98f002a-2718-48c7-8a9a-8905c76434e9-kube-api-access-dtnwc\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:17:55.597389 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.597359 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtnwc\" (UniqueName: \"kubernetes.io/projected/b98f002a-2718-48c7-8a9a-8905c76434e9-kube-api-access-dtnwc\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:17:55.689215 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.689147 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9l9\" (UniqueName: \"kubernetes.io/projected/ffa9c82e-220f-4b99-bf40-4136da9bd925-kube-api-access-hl9l9\") pod \"volume-data-source-validator-549c6d54cf-5msdh\" (UID: \"ffa9c82e-220f-4b99-bf40-4136da9bd925\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-5msdh" Jul 28 17:17:55.698724 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.698700 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl9l9\" (UniqueName: \"kubernetes.io/projected/ffa9c82e-220f-4b99-bf40-4136da9bd925-kube-api-access-hl9l9\") pod \"volume-data-source-validator-549c6d54cf-5msdh\" (UID: \"ffa9c82e-220f-4b99-bf40-4136da9bd925\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-5msdh" Jul 28 17:17:55.798638 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.798607 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-5msdh" Jul 28 17:17:55.935499 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:55.935469 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-5msdh"] Jul 28 17:17:55.938580 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:55.938552 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa9c82e_220f_4b99_bf40_4136da9bd925.slice/crio-fe2daac9ef94a369b16095d92664ed50f8388d47ea85d25307b72e792cab1e3e WatchSource:0}: Error finding container fe2daac9ef94a369b16095d92664ed50f8388d47ea85d25307b72e792cab1e3e: Status 404 returned error can't find the container with id fe2daac9ef94a369b16095d92664ed50f8388d47ea85d25307b72e792cab1e3e Jul 28 17:17:56.091449 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:56.091419 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:17:56.091595 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:56.091560 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:17:56.091646 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:56.091635 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls podName:b98f002a-2718-48c7-8a9a-8905c76434e9 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:57.091619267 +0000 UTC m=+54.464167841 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls") pod "cluster-samples-operator-6748489656-srchs" (UID: "b98f002a-2718-48c7-8a9a-8905c76434e9") : secret "samples-operator-tls" not found Jul 28 17:17:56.190167 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:56.190097 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4vvsd_09009b39-09ab-475e-bc10-1c0f559e4a07/dns-node-resolver/0.log" Jul 28 17:17:56.353217 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:56.353185 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-5msdh" event={"ID":"ffa9c82e-220f-4b99-bf40-4136da9bd925","Type":"ContainerStarted","Data":"fe2daac9ef94a369b16095d92664ed50f8388d47ea85d25307b72e792cab1e3e"} Jul 28 17:17:56.495077 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:56.494974 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:56.495077 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:56.495069 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:56.495636 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:56.495143 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:17:56.495636 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:56.495209 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls podName:eae07057-b8a0-4138-88fd-0ba713b04676 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:58.495192026 +0000 UTC m=+55.867740600 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9cghg" (UID: "eae07057-b8a0-4138-88fd-0ba713b04676") : secret "insights-runtime-extractor-tls" not found Jul 28 17:17:56.495636 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:56.495143 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:17:56.495636 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:56.495319 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm podName:eae07057-b8a0-4138-88fd-0ba713b04676 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:58.495295021 +0000 UTC m=+55.867843609 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9cghg" (UID: "eae07057-b8a0-4138-88fd-0ba713b04676") : configmap "kube-rbac-proxy" not found Jul 28 17:17:56.990025 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:56.989993 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87p9h_c795a38b-13c3-4b44-ae37-9de0fd450a00/node-ca/0.log" Jul 28 17:17:57.101348 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.101307 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:17:57.101545 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:57.101456 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:17:57.101545 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:57.101523 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls podName:b98f002a-2718-48c7-8a9a-8905c76434e9 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:59.101506728 +0000 UTC m=+56.474055302 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls") pod "cluster-samples-operator-6748489656-srchs" (UID: "b98f002a-2718-48c7-8a9a-8905c76434e9") : secret "samples-operator-tls" not found Jul 28 17:17:57.357334 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.357302 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-5msdh" event={"ID":"ffa9c82e-220f-4b99-bf40-4136da9bd925","Type":"ContainerStarted","Data":"13799ae9556b9913774e4fa0f0fc4447477f18969c64b597dd4caa69402dee9f"} Jul 28 17:17:57.373429 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.373383 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-5msdh" podStartSLOduration=1.161233702 podStartE2EDuration="2.373367601s" totalImagesPullingTime="1.212133899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:57.372701892 +0000 UTC m=+54.745250488" watchObservedRunningTime="2026-07-28 17:17:57.373367601 +0000 UTC m=+54.745916198" Jul 28 17:17:57.394305 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.394278 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-862sz"] Jul 28 17:17:57.396150 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.396134 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.398568 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.398550 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 28 17:17:57.398707 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.398690 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 28 17:17:57.398812 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.398690 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:17:57.398874 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.398815 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dggr9\"" Jul 28 17:17:57.398874 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.398836 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 28 17:17:57.403499 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.403484 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 28 17:17:57.407982 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.407962 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-862sz"] Jul 28 17:17:57.505024 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.504994 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae78b679-6ed6-4889-a920-585392ae4f8e-serving-cert\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.505339 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.505036 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ae78b679-6ed6-4889-a920-585392ae4f8e-config\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.505339 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.505077 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae78b679-6ed6-4889-a920-585392ae4f8e-trusted-ca\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.505339 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.505130 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4292n\" (UniqueName: \"kubernetes.io/projected/ae78b679-6ed6-4889-a920-585392ae4f8e-kube-api-access-4292n\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.606458 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.606423 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae78b679-6ed6-4889-a920-585392ae4f8e-serving-cert\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.606614 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.606470 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ae78b679-6ed6-4889-a920-585392ae4f8e-config\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.606614 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.606500 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae78b679-6ed6-4889-a920-585392ae4f8e-trusted-ca\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.606614 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.606523 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4292n\" (UniqueName: \"kubernetes.io/projected/ae78b679-6ed6-4889-a920-585392ae4f8e-kube-api-access-4292n\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.607194 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.607166 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ae78b679-6ed6-4889-a920-585392ae4f8e-config\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.607518 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.607466 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ae78b679-6ed6-4889-a920-585392ae4f8e-trusted-ca\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.608843 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.608823 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae78b679-6ed6-4889-a920-585392ae4f8e-serving-cert\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.615313 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.615292 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4292n\" (UniqueName: \"kubernetes.io/projected/ae78b679-6ed6-4889-a920-585392ae4f8e-kube-api-access-4292n\") pod \"console-operator-584dcb6b54-862sz\" (UID: \"ae78b679-6ed6-4889-a920-585392ae4f8e\") " pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.705065 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.705026 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:17:57.833812 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.833763 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-862sz"] Jul 28 17:17:57.836912 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:17:57.836880 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae78b679_6ed6_4889_a920_585392ae4f8e.slice/crio-0095263aa1839ab0dd19e874ec2c8740b594580a89f3df3d4559a8611b046a0a WatchSource:0}: Error finding container 0095263aa1839ab0dd19e874ec2c8740b594580a89f3df3d4559a8611b046a0a: Status 404 returned error can't find the container with id 0095263aa1839ab0dd19e874ec2c8740b594580a89f3df3d4559a8611b046a0a Jul 28 17:17:57.995820 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:57.995722 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xg2v2_db7b2d75-c2fa-4f26-986e-ce1164ae5b67/serve-healthcheck-canary/0.log" Jul 28 17:17:58.360580 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:58.360541 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" event={"ID":"ae78b679-6ed6-4889-a920-585392ae4f8e","Type":"ContainerStarted","Data":"0095263aa1839ab0dd19e874ec2c8740b594580a89f3df3d4559a8611b046a0a"} Jul 28 17:17:58.513067 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:58.513035 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:58.513477 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:58.513101 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:17:58.513477 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:58.513189 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:17:58.513477 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:58.513203 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:17:58.513477 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:58.513252 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls podName:eae07057-b8a0-4138-88fd-0ba713b04676 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:02.513238621 +0000 UTC m=+59.885787198 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9cghg" (UID: "eae07057-b8a0-4138-88fd-0ba713b04676") : secret "insights-runtime-extractor-tls" not found Jul 28 17:17:58.513477 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:58.513345 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm podName:eae07057-b8a0-4138-88fd-0ba713b04676 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:02.513317059 +0000 UTC m=+59.885865649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9cghg" (UID: "eae07057-b8a0-4138-88fd-0ba713b04676") : configmap "kube-rbac-proxy" not found Jul 28 17:17:59.118467 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:59.118427 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:17:59.118641 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:59.118600 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:17:59.118699 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:59.118681 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls podName:b98f002a-2718-48c7-8a9a-8905c76434e9 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:03.118658016 +0000 UTC m=+60.491206633 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls") pod "cluster-samples-operator-6748489656-srchs" (UID: "b98f002a-2718-48c7-8a9a-8905c76434e9") : secret "samples-operator-tls" not found Jul 28 17:17:59.219201 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:17:59.219166 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:17:59.219370 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:59.219354 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:17:59.219439 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:17:59.219426 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls podName:5d7e904f-06b5-4dfb-ac37-a97b38ace547 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:07.219407287 +0000 UTC m=+64.591955865 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sfqvm" (UID: "5d7e904f-06b5-4dfb-ac37-a97b38ace547") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:18:00.368979 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:00.368945 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/0.log" Jul 28 17:18:00.369445 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:00.368987 2576 generic.go:356] "Generic (PLEG): container finished" podID="ae78b679-6ed6-4889-a920-585392ae4f8e" containerID="c5075960eeb1d5a2aff63affa206c0d569275ec96ab054b0c54274cb7a5dea83" exitCode=255 Jul 28 17:18:00.369445 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:00.369040 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" event={"ID":"ae78b679-6ed6-4889-a920-585392ae4f8e","Type":"ContainerDied","Data":"c5075960eeb1d5a2aff63affa206c0d569275ec96ab054b0c54274cb7a5dea83"} Jul 28 17:18:00.369445 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:00.369307 2576 scope.go:117] "RemoveContainer" containerID="c5075960eeb1d5a2aff63affa206c0d569275ec96ab054b0c54274cb7a5dea83" Jul 28 17:18:01.373502 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:01.373477 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:18:01.373925 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:01.373901 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/0.log" Jul 28 17:18:01.373990 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:01.373940 2576 generic.go:356] "Generic (PLEG): container finished" podID="ae78b679-6ed6-4889-a920-585392ae4f8e" containerID="44bc356fcf143d0d28df7be1ad06eb77297dcc177db0407365a26a32b98f4cd5" exitCode=255 Jul 28 17:18:01.374028 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:01.373993 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" event={"ID":"ae78b679-6ed6-4889-a920-585392ae4f8e","Type":"ContainerDied","Data":"44bc356fcf143d0d28df7be1ad06eb77297dcc177db0407365a26a32b98f4cd5"} Jul 28 17:18:01.374061 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:01.374046 2576 scope.go:117] "RemoveContainer" containerID="c5075960eeb1d5a2aff63affa206c0d569275ec96ab054b0c54274cb7a5dea83" Jul 28 17:18:01.374258 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:01.374236 2576 scope.go:117] "RemoveContainer" containerID="44bc356fcf143d0d28df7be1ad06eb77297dcc177db0407365a26a32b98f4cd5" Jul 28 17:18:01.374452 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:01.374428 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-862sz_openshift-console-operator(ae78b679-6ed6-4889-a920-585392ae4f8e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" podUID="ae78b679-6ed6-4889-a920-585392ae4f8e" Jul 28 17:18:02.378083 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:02.378058 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:18:02.378429 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:02.378360 2576 scope.go:117] "RemoveContainer" containerID="44bc356fcf143d0d28df7be1ad06eb77297dcc177db0407365a26a32b98f4cd5" Jul 28 17:18:02.378569 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:02.378545 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-862sz_openshift-console-operator(ae78b679-6ed6-4889-a920-585392ae4f8e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" podUID="ae78b679-6ed6-4889-a920-585392ae4f8e" Jul 28 17:18:02.548755 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:02.548718 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:18:02.548965 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:02.548774 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:18:02.548965 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:02.548884 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:18:02.548965 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:02.548921 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:18:02.548965 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:02.548947 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm podName:eae07057-b8a0-4138-88fd-0ba713b04676 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:10.548930595 +0000 UTC m=+67.921479186 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9cghg" (UID: "eae07057-b8a0-4138-88fd-0ba713b04676") : configmap "kube-rbac-proxy" not found Jul 28 17:18:02.549117 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:02.548988 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls podName:eae07057-b8a0-4138-88fd-0ba713b04676 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:10.548969038 +0000 UTC m=+67.921517615 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9cghg" (UID: "eae07057-b8a0-4138-88fd-0ba713b04676") : secret "insights-runtime-extractor-tls" not found Jul 28 17:18:03.155271 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:03.155243 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:18:03.155406 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:03.155381 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:18:03.155454 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:03.155438 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls podName:b98f002a-2718-48c7-8a9a-8905c76434e9 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:11.155423423 +0000 UTC m=+68.527971996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls") pod "cluster-samples-operator-6748489656-srchs" (UID: "b98f002a-2718-48c7-8a9a-8905c76434e9") : secret "samples-operator-tls" not found Jul 28 17:18:03.300840 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:03.300815 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c29m8" Jul 28 17:18:07.287310 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:07.287261 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:18:07.287716 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:07.287382 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:18:07.287716 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:07.287436 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls podName:5d7e904f-06b5-4dfb-ac37-a97b38ace547 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:23.287422359 +0000 UTC m=+80.659970933 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sfqvm" (UID: "5d7e904f-06b5-4dfb-ac37-a97b38ace547") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:18:07.705925 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:07.705830 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:18:07.705925 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:07.705901 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:18:07.706230 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:07.706218 2576 scope.go:117] "RemoveContainer" containerID="44bc356fcf143d0d28df7be1ad06eb77297dcc177db0407365a26a32b98f4cd5" Jul 28 17:18:07.706387 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:07.706367 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-862sz_openshift-console-operator(ae78b679-6ed6-4889-a920-585392ae4f8e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" podUID="ae78b679-6ed6-4889-a920-585392ae4f8e" Jul 28 17:18:08.784858 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.784820 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-h6rzn"] Jul 28 17:18:08.787690 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.787674 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" Jul 28 17:18:08.790253 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.790215 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 28 17:18:08.790371 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.790303 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 28 17:18:08.791438 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.791421 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-sl2kd\"" Jul 28 17:18:08.791561 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.791448 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 28 17:18:08.791561 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.791485 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 28 17:18:08.797942 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.797922 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-h6rzn"] Jul 28 17:18:08.799165 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.799145 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:18:08.801463 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.801443 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:18:08.811621 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.811598 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c71f50e-2184-425b-9d24-0e2c2aebcaac-original-pull-secret\") pod \"global-pull-secret-syncer-tjmr6\" (UID: \"6c71f50e-2184-425b-9d24-0e2c2aebcaac\") " pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:18:08.899864 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.899827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:18:08.900017 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.899896 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/52a2b645-1c56-4b35-abe1-05ea5e4ef0c4-signing-key\") pod \"service-ca-658d7b7c78-h6rzn\" (UID: \"52a2b645-1c56-4b35-abe1-05ea5e4ef0c4\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" Jul 28 17:18:08.900017 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.899927 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p9l4\" (UniqueName: \"kubernetes.io/projected/52a2b645-1c56-4b35-abe1-05ea5e4ef0c4-kube-api-access-7p9l4\") pod \"service-ca-658d7b7c78-h6rzn\" (UID: \"52a2b645-1c56-4b35-abe1-05ea5e4ef0c4\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" Jul 28 17:18:08.900017 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.899948 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/52a2b645-1c56-4b35-abe1-05ea5e4ef0c4-signing-cabundle\") pod \"service-ca-658d7b7c78-h6rzn\" (UID: \"52a2b645-1c56-4b35-abe1-05ea5e4ef0c4\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" Jul 28 17:18:08.900017 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:08.899964 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:18:08.900149 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:08.900025 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls podName:0a20f6b5-d596-481d-8de8-edb57100e326 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:40.900009245 +0000 UTC m=+98.272557821 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls") pod "dns-default-tx79n" (UID: "0a20f6b5-d596-481d-8de8-edb57100e326") : secret "dns-default-metrics-tls" not found Jul 28 17:18:08.900149 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.900084 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:18:08.902437 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:08.902421 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:18:08.910509 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:08.910493 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 28 17:18:08.910565 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:08.910547 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs podName:e761eac4-98d4-494a-be69-82b2f0e6e4d4 nodeName:}" failed. No retries permitted until 2026-07-28 17:19:12.910534248 +0000 UTC m=+130.283082822 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs") pod "network-metrics-daemon-s9qjh" (UID: "e761eac4-98d4-494a-be69-82b2f0e6e4d4") : secret "metrics-daemon-secret" not found Jul 28 17:18:09.001466 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.001429 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j66z7\" (UniqueName: \"kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7\") pod \"network-check-target-5g2mj\" (UID: \"4701c1b9-f966-41c6-bd60-86d9fc2bf11e\") " pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:18:09.001602 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.001478 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/52a2b645-1c56-4b35-abe1-05ea5e4ef0c4-signing-key\") pod \"service-ca-658d7b7c78-h6rzn\" (UID: \"52a2b645-1c56-4b35-abe1-05ea5e4ef0c4\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" Jul 28 17:18:09.001602 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.001509 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p9l4\" (UniqueName: \"kubernetes.io/projected/52a2b645-1c56-4b35-abe1-05ea5e4ef0c4-kube-api-access-7p9l4\") pod \"service-ca-658d7b7c78-h6rzn\" (UID: \"52a2b645-1c56-4b35-abe1-05ea5e4ef0c4\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" Jul 28 17:18:09.001602 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.001525 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/52a2b645-1c56-4b35-abe1-05ea5e4ef0c4-signing-cabundle\") pod \"service-ca-658d7b7c78-h6rzn\" (UID: \"52a2b645-1c56-4b35-abe1-05ea5e4ef0c4\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" Jul 28 17:18:09.002255 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.002237 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/52a2b645-1c56-4b35-abe1-05ea5e4ef0c4-signing-cabundle\") pod \"service-ca-658d7b7c78-h6rzn\" (UID: \"52a2b645-1c56-4b35-abe1-05ea5e4ef0c4\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" Jul 28 17:18:09.004037 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.004006 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/52a2b645-1c56-4b35-abe1-05ea5e4ef0c4-signing-key\") pod \"service-ca-658d7b7c78-h6rzn\" (UID: \"52a2b645-1c56-4b35-abe1-05ea5e4ef0c4\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" Jul 28 17:18:09.004136 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.004095 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 17:18:09.010430 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.010411 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p9l4\" (UniqueName: \"kubernetes.io/projected/52a2b645-1c56-4b35-abe1-05ea5e4ef0c4-kube-api-access-7p9l4\") pod \"service-ca-658d7b7c78-h6rzn\" (UID: \"52a2b645-1c56-4b35-abe1-05ea5e4ef0c4\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" Jul 28 17:18:09.014428 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.014410 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 17:18:09.024589 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.024564 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j66z7\" (UniqueName: \"kubernetes.io/projected/4701c1b9-f966-41c6-bd60-86d9fc2bf11e-kube-api-access-j66z7\") pod \"network-check-target-5g2mj\" (UID: \"4701c1b9-f966-41c6-bd60-86d9fc2bf11e\") " pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:18:09.055085 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.055035 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qzphg\"" Jul 28 17:18:09.059251 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.059238 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmr6" Jul 28 17:18:09.062771 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.062757 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:18:09.096966 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.096928 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" Jul 28 17:18:09.204472 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.204448 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tjmr6"] Jul 28 17:18:09.207916 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:09.207887 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c71f50e_2184_425b_9d24_0e2c2aebcaac.slice/crio-810c3e8e75b9e7a9f437cf6181a678188b27e00067d5371baf3348dd008388c4 WatchSource:0}: Error finding container 810c3e8e75b9e7a9f437cf6181a678188b27e00067d5371baf3348dd008388c4: Status 404 returned error can't find the container with id 810c3e8e75b9e7a9f437cf6181a678188b27e00067d5371baf3348dd008388c4 Jul 28 17:18:09.224329 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.224302 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5g2mj"] Jul 28 17:18:09.228849 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:09.228822 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4701c1b9_f966_41c6_bd60_86d9fc2bf11e.slice/crio-82d596377ef8354377e402f058feb12b1eb96570a6b8e51a88707118629976c6 WatchSource:0}: Error finding container 82d596377ef8354377e402f058feb12b1eb96570a6b8e51a88707118629976c6: Status 404 returned error can't find the container with id 82d596377ef8354377e402f058feb12b1eb96570a6b8e51a88707118629976c6 Jul 28 17:18:09.251342 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.251319 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-h6rzn"] Jul 28 17:18:09.254699 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:09.254660 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a2b645_1c56_4b35_abe1_05ea5e4ef0c4.slice/crio-56ed1fbdf2abac0498bfb188dd22018f62be52d0ab8bb609c9c7fc5ed2c14ae0 WatchSource:0}: Error finding container 56ed1fbdf2abac0498bfb188dd22018f62be52d0ab8bb609c9c7fc5ed2c14ae0: Status 404 returned error can't find the container with id 56ed1fbdf2abac0498bfb188dd22018f62be52d0ab8bb609c9c7fc5ed2c14ae0 Jul 28 17:18:09.395093 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.395003 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5g2mj" event={"ID":"4701c1b9-f966-41c6-bd60-86d9fc2bf11e","Type":"ContainerStarted","Data":"82d596377ef8354377e402f058feb12b1eb96570a6b8e51a88707118629976c6"} Jul 28 17:18:09.396085 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.396062 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tjmr6" event={"ID":"6c71f50e-2184-425b-9d24-0e2c2aebcaac","Type":"ContainerStarted","Data":"810c3e8e75b9e7a9f437cf6181a678188b27e00067d5371baf3348dd008388c4"} Jul 28 17:18:09.396987 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:09.396970 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" event={"ID":"52a2b645-1c56-4b35-abe1-05ea5e4ef0c4","Type":"ContainerStarted","Data":"56ed1fbdf2abac0498bfb188dd22018f62be52d0ab8bb609c9c7fc5ed2c14ae0"} Jul 28 17:18:10.616512 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:10.616469 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:18:10.617032 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:10.616550 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:18:10.617032 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:10.616666 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:18:10.617032 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:10.616776 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls podName:eae07057-b8a0-4138-88fd-0ba713b04676 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:26.616754188 +0000 UTC m=+83.989302764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9cghg" (UID: "eae07057-b8a0-4138-88fd-0ba713b04676") : secret "insights-runtime-extractor-tls" not found Jul 28 17:18:10.617249 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:10.617225 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eae07057-b8a0-4138-88fd-0ba713b04676-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:18:11.222400 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:11.221746 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:18:11.222400 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:11.222016 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:18:11.222400 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:11.222102 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls podName:b98f002a-2718-48c7-8a9a-8905c76434e9 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:27.222082048 +0000 UTC m=+84.594630625 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls") pod "cluster-samples-operator-6748489656-srchs" (UID: "b98f002a-2718-48c7-8a9a-8905c76434e9") : secret "samples-operator-tls" not found Jul 28 17:18:14.414263 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:14.414155 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" event={"ID":"52a2b645-1c56-4b35-abe1-05ea5e4ef0c4","Type":"ContainerStarted","Data":"2c381484c9fd402966ecdd38e2d4672f0b5ca5e08422019fef242cdd110f36c2"} Jul 28 17:18:14.415761 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:14.415732 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5g2mj" event={"ID":"4701c1b9-f966-41c6-bd60-86d9fc2bf11e","Type":"ContainerStarted","Data":"05c0b11e6de07a4d9c262a8e323da6161abebdd46c5e8e3c3889a54e5ab9ca92"} Jul 28 17:18:14.415903 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:14.415819 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:18:14.417116 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:14.417097 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tjmr6" event={"ID":"6c71f50e-2184-425b-9d24-0e2c2aebcaac","Type":"ContainerStarted","Data":"d97ad3a06d53163a01d802778a2c784e26c52746828b013687ecfce9cdd34b27"} Jul 28 17:18:14.431448 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:14.431412 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-h6rzn" podStartSLOduration=1.638228885 podStartE2EDuration="6.431399348s" totalImagesPullingTime="4.793170463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:14.431052614 +0000 UTC m=+71.803601212" watchObservedRunningTime="2026-07-28 17:18:14.431399348 +0000 UTC m=+71.803947995" Jul 28 17:18:14.447031 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:14.446986 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tjmr6" podStartSLOduration=66.590985427 podStartE2EDuration="1m11.446970092s" totalImagesPullingTime="4.855984665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:14.446278284 +0000 UTC m=+71.818826891" watchObservedRunningTime="2026-07-28 17:18:14.446970092 +0000 UTC m=+71.819518690" Jul 28 17:18:14.462132 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:14.462093 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5g2mj" podStartSLOduration=66.629737301 podStartE2EDuration="1m11.462077719s" totalImagesPullingTime="4.832340418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:14.461949396 +0000 UTC m=+71.834498003" watchObservedRunningTime="2026-07-28 17:18:14.462077719 +0000 UTC m=+71.834626317" Jul 28 17:18:23.139857 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:23.139827 2576 scope.go:117] "RemoveContainer" containerID="44bc356fcf143d0d28df7be1ad06eb77297dcc177db0407365a26a32b98f4cd5" Jul 28 17:18:23.315320 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:23.315287 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:18:23.317746 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:23.317711 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d7e904f-06b5-4dfb-ac37-a97b38ace547-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sfqvm\" (UID: \"5d7e904f-06b5-4dfb-ac37-a97b38ace547\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:18:23.450535 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:23.450459 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:18:23.450535 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:23.450526 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" event={"ID":"ae78b679-6ed6-4889-a920-585392ae4f8e","Type":"ContainerStarted","Data":"ed0e3d1be66b4d4f25e0dc25dac7bdff0bc706b4a0678b95bdb9eead76fcac42"} Jul 28 17:18:23.450861 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:23.450835 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:18:23.469537 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:23.469484 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" podStartSLOduration=24.625490842 podStartE2EDuration="26.469468459s" totalImagesPullingTime="1.843977617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:23.468984135 +0000 UTC m=+80.841532728" watchObservedRunningTime="2026-07-28 17:18:23.469468459 +0000 UTC m=+80.842017058" Jul 28 17:18:23.530474 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:23.530448 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bc2pz\"" Jul 28 17:18:23.539105 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:23.539085 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" Jul 28 17:18:23.669708 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:23.669676 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm"] Jul 28 17:18:23.672662 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:23.672623 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7e904f_06b5_4dfb_ac37_a97b38ace547.slice/crio-6d7d4cd354211f7412b340ff549f3835c87b320d259e4f96e8e55680c108a02b WatchSource:0}: Error finding container 6d7d4cd354211f7412b340ff549f3835c87b320d259e4f96e8e55680c108a02b: Status 404 returned error can't find the container with id 6d7d4cd354211f7412b340ff549f3835c87b320d259e4f96e8e55680c108a02b Jul 28 17:18:24.100320 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:24.100290 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-862sz" Jul 28 17:18:24.457117 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:24.457033 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" event={"ID":"5d7e904f-06b5-4dfb-ac37-a97b38ace547","Type":"ContainerStarted","Data":"6d7d4cd354211f7412b340ff549f3835c87b320d259e4f96e8e55680c108a02b"} Jul 28 17:18:25.462261 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:25.462223 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" event={"ID":"5d7e904f-06b5-4dfb-ac37-a97b38ace547","Type":"ContainerStarted","Data":"dc712064eafcd9a7541a441320a2ac1bc9f41724ca1f707a07a509d77a5e2b0e"} Jul 28 17:18:25.480763 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:25.480706 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sfqvm" podStartSLOduration=32.790779698 podStartE2EDuration="34.480687244s" totalImagesPullingTime="1.689907546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:25.479858246 +0000 UTC m=+82.852406842" watchObservedRunningTime="2026-07-28 17:18:25.480687244 +0000 UTC m=+82.853235851" Jul 28 17:18:26.644615 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:26.644578 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:18:26.646923 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:26.646897 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eae07057-b8a0-4138-88fd-0ba713b04676-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9cghg\" (UID: \"eae07057-b8a0-4138-88fd-0ba713b04676\") " pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:18:26.835470 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:26.835440 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mc5b9\"" Jul 28 17:18:26.843456 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:26.843425 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9cghg" Jul 28 17:18:26.979612 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:26.979583 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9cghg"] Jul 28 17:18:26.982939 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:26.982913 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeae07057_b8a0_4138_88fd_0ba713b04676.slice/crio-680dbcd365347b5e4a495900a1da2281c27b5512e139215cd51aba62c3388f81 WatchSource:0}: Error finding container 680dbcd365347b5e4a495900a1da2281c27b5512e139215cd51aba62c3388f81: Status 404 returned error can't find the container with id 680dbcd365347b5e4a495900a1da2281c27b5512e139215cd51aba62c3388f81 Jul 28 17:18:27.249470 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:27.249389 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:18:27.251709 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:27.251678 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b98f002a-2718-48c7-8a9a-8905c76434e9-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-srchs\" (UID: \"b98f002a-2718-48c7-8a9a-8905c76434e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:18:27.472363 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:27.472318 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9cghg" event={"ID":"eae07057-b8a0-4138-88fd-0ba713b04676","Type":"ContainerStarted","Data":"680dbcd365347b5e4a495900a1da2281c27b5512e139215cd51aba62c3388f81"} Jul 28 17:18:27.490231 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:27.490206 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zwcz8\"" Jul 28 17:18:27.498327 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:27.498300 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" Jul 28 17:18:27.657688 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:27.657615 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs"] Jul 28 17:18:28.477839 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:28.477800 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" event={"ID":"b98f002a-2718-48c7-8a9a-8905c76434e9","Type":"ContainerStarted","Data":"e676f98debea4da2b8e0311a57e9ec6e113a4fa365e7e99198f26df4eaff7970"} Jul 28 17:18:29.048909 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.048373 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b7f4fb6c9-l874l"] Jul 28 17:18:29.056975 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.056933 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2"] Jul 28 17:18:29.057109 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.057037 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.059689 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.059643 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4mv5p\"" Jul 28 17:18:29.059689 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.059692 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 17:18:29.059880 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.059735 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 17:18:29.059880 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.059649 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 17:18:29.060589 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.060567 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2" Jul 28 17:18:29.063305 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.063283 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xr7bh\"" Jul 28 17:18:29.063630 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.063614 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 28 17:18:29.065995 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.065916 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 17:18:29.067922 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.067903 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2"] Jul 28 17:18:29.068916 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.068898 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b7f4fb6c9-l874l"] Jul 28 17:18:29.165150 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.165118 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f128ad08-a030-47de-ae1b-18bbd74e505e-bound-sa-token\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.165342 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.165167 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f128ad08-a030-47de-ae1b-18bbd74e505e-trusted-ca\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.165342 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.165208 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt6b9\" (UniqueName: \"kubernetes.io/projected/f128ad08-a030-47de-ae1b-18bbd74e505e-kube-api-access-qt6b9\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.165342 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.165264 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f128ad08-a030-47de-ae1b-18bbd74e505e-registry-certificates\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.165342 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.165303 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f128ad08-a030-47de-ae1b-18bbd74e505e-ca-trust-extracted\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.165544 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.165341 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f128ad08-a030-47de-ae1b-18bbd74e505e-registry-tls\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.165544 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.165367 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f128ad08-a030-47de-ae1b-18bbd74e505e-image-registry-private-configuration\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.165544 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.165388 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/76c12730-9f6e-451f-a485-db4456cf3e1d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qpft2\" (UID: \"76c12730-9f6e-451f-a485-db4456cf3e1d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2" Jul 28 17:18:29.165544 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.165406 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f128ad08-a030-47de-ae1b-18bbd74e505e-installation-pull-secrets\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.266556 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.266523 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f128ad08-a030-47de-ae1b-18bbd74e505e-trusted-ca\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.266730 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.266596 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt6b9\" (UniqueName: \"kubernetes.io/projected/f128ad08-a030-47de-ae1b-18bbd74e505e-kube-api-access-qt6b9\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.267011 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.266991 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f128ad08-a030-47de-ae1b-18bbd74e505e-registry-certificates\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.267177 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.267057 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f128ad08-a030-47de-ae1b-18bbd74e505e-ca-trust-extracted\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.267242 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.267196 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f128ad08-a030-47de-ae1b-18bbd74e505e-registry-tls\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.267300 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.267240 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f128ad08-a030-47de-ae1b-18bbd74e505e-image-registry-private-configuration\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.267300 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.267283 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/76c12730-9f6e-451f-a485-db4456cf3e1d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qpft2\" (UID: \"76c12730-9f6e-451f-a485-db4456cf3e1d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2" Jul 28 17:18:29.267399 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.267315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f128ad08-a030-47de-ae1b-18bbd74e505e-installation-pull-secrets\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.267399 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.267361 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f128ad08-a030-47de-ae1b-18bbd74e505e-bound-sa-token\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.267399 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.267387 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f128ad08-a030-47de-ae1b-18bbd74e505e-ca-trust-extracted\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.268449 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.267752 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f128ad08-a030-47de-ae1b-18bbd74e505e-trusted-ca\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.268449 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.267877 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f128ad08-a030-47de-ae1b-18bbd74e505e-registry-certificates\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.271186 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.271161 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f128ad08-a030-47de-ae1b-18bbd74e505e-image-registry-private-configuration\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.271719 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.271693 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/76c12730-9f6e-451f-a485-db4456cf3e1d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qpft2\" (UID: \"76c12730-9f6e-451f-a485-db4456cf3e1d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2" Jul 28 17:18:29.272167 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.272145 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f128ad08-a030-47de-ae1b-18bbd74e505e-registry-tls\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.272580 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.272557 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f128ad08-a030-47de-ae1b-18bbd74e505e-installation-pull-secrets\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.277079 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.277038 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt6b9\" (UniqueName: \"kubernetes.io/projected/f128ad08-a030-47de-ae1b-18bbd74e505e-kube-api-access-qt6b9\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.278067 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.278034 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f128ad08-a030-47de-ae1b-18bbd74e505e-bound-sa-token\") pod \"image-registry-b7f4fb6c9-l874l\" (UID: \"f128ad08-a030-47de-ae1b-18bbd74e505e\") " pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.372434 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.372340 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:29.379742 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:29.379717 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2" Jul 28 17:18:32.495622 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:32.494502 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" event={"ID":"b98f002a-2718-48c7-8a9a-8905c76434e9","Type":"ContainerStarted","Data":"5435660794df870c10c9a495c653854c24f54dcd7c80f6d0f1178fdf9e7c2e90"} Jul 28 17:18:32.547862 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:32.547769 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b7f4fb6c9-l874l"] Jul 28 17:18:32.551897 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:32.551857 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf128ad08_a030_47de_ae1b_18bbd74e505e.slice/crio-ca997d6032ac840fc08c99bface5af1f8c4181df82a2e7811445efe9615cdb6a WatchSource:0}: Error finding container ca997d6032ac840fc08c99bface5af1f8c4181df82a2e7811445efe9615cdb6a: Status 404 returned error can't find the container with id ca997d6032ac840fc08c99bface5af1f8c4181df82a2e7811445efe9615cdb6a Jul 28 17:18:32.559648 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:32.559628 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2"] Jul 28 17:18:32.564045 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:32.563998 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76c12730_9f6e_451f_a485_db4456cf3e1d.slice/crio-d55cf682b134251f0bc295596ade91e7bdae0d986da5b18fa3dda3560c8be04e WatchSource:0}: Error finding container d55cf682b134251f0bc295596ade91e7bdae0d986da5b18fa3dda3560c8be04e: Status 404 returned error can't find the container with id d55cf682b134251f0bc295596ade91e7bdae0d986da5b18fa3dda3560c8be04e Jul 28 17:18:33.501021 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:33.500941 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" event={"ID":"b98f002a-2718-48c7-8a9a-8905c76434e9","Type":"ContainerStarted","Data":"551dd2cef7d9b9ad23f2ec924fc26ae9e03e0807ceb15955a702f70632db8823"} Jul 28 17:18:33.502633 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:33.502576 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2" event={"ID":"76c12730-9f6e-451f-a485-db4456cf3e1d","Type":"ContainerStarted","Data":"d55cf682b134251f0bc295596ade91e7bdae0d986da5b18fa3dda3560c8be04e"} Jul 28 17:18:33.504369 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:33.504318 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9cghg" event={"ID":"eae07057-b8a0-4138-88fd-0ba713b04676","Type":"ContainerStarted","Data":"4cc17a8bb3dabe926693d6d9ee182a3b6d58e97be54c490eae660bfe8cbcbf10"} Jul 28 17:18:33.506915 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:33.506889 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" event={"ID":"f128ad08-a030-47de-ae1b-18bbd74e505e","Type":"ContainerStarted","Data":"78ffe6a354e3d4241a50036d28b042c6ca63ed0d3a987f745e49cebda873fd6b"} Jul 28 17:18:33.507015 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:33.506935 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" event={"ID":"f128ad08-a030-47de-ae1b-18bbd74e505e","Type":"ContainerStarted","Data":"ca997d6032ac840fc08c99bface5af1f8c4181df82a2e7811445efe9615cdb6a"} Jul 28 17:18:33.507091 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:33.507067 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:33.522019 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:33.521966 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-srchs" podStartSLOduration=33.874090243 podStartE2EDuration="38.521947904s" totalImagesPullingTime="4.647857661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:33.521436428 +0000 UTC m=+90.893985025" watchObservedRunningTime="2026-07-28 17:18:33.521947904 +0000 UTC m=+90.894496482" Jul 28 17:18:33.543478 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:33.542324 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" podStartSLOduration=4.542304823 podStartE2EDuration="4.542304823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:33.541550524 +0000 UTC m=+90.914099123" watchObservedRunningTime="2026-07-28 17:18:33.542304823 +0000 UTC m=+90.914853423" Jul 28 17:18:34.512250 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:34.512210 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2" event={"ID":"76c12730-9f6e-451f-a485-db4456cf3e1d","Type":"ContainerStarted","Data":"40994e396e1db1bfe708194ed42140b8ff03dedbcc6699a7643b6a453b020a8f"} Jul 28 17:18:34.534679 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:34.534633 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2" podStartSLOduration=4.373967901 podStartE2EDuration="5.534615022s" totalImagesPullingTime="1.160647121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:34.534306124 +0000 UTC m=+91.906854720" watchObservedRunningTime="2026-07-28 17:18:34.534615022 +0000 UTC m=+91.907163618" Jul 28 17:18:35.516387 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:35.516345 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2" Jul 28 17:18:35.521695 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:35.521667 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qpft2" Jul 28 17:18:36.522294 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:36.522262 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9cghg" event={"ID":"eae07057-b8a0-4138-88fd-0ba713b04676","Type":"ContainerStarted","Data":"4802f04feccd779df5f4ebd30a92821ae74b4a07390194ffbf846b009d2f3188"} Jul 28 17:18:40.538940 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:40.538907 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9cghg" event={"ID":"eae07057-b8a0-4138-88fd-0ba713b04676","Type":"ContainerStarted","Data":"7d255f1319f8c66841368c9dd87174826433eb542296246ad38af0c91037c8ff"} Jul 28 17:18:40.559691 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:40.559640 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9cghg" podStartSLOduration=34.182198054 podStartE2EDuration="46.559623341s" totalImagesPullingTime="12.377425287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:40.557888267 +0000 UTC m=+97.930436862" watchObservedRunningTime="2026-07-28 17:18:40.559623341 +0000 UTC m=+97.932171937" Jul 28 17:18:40.966213 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:40.966119 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:18:40.968403 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:40.968373 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a20f6b5-d596-481d-8de8-edb57100e326-metrics-tls\") pod \"dns-default-tx79n\" (UID: \"0a20f6b5-d596-481d-8de8-edb57100e326\") " pod="openshift-dns/dns-default-tx79n" Jul 28 17:18:41.212720 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.212683 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qjb4d\"" Jul 28 17:18:41.220901 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.220833 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tx79n" Jul 28 17:18:41.361861 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.361828 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8"] Jul 28 17:18:41.366459 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.366437 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.369638 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.369613 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kvgln\"" Jul 28 17:18:41.371014 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.370993 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 28 17:18:41.371131 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.371012 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 28 17:18:41.371401 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.371381 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 17:18:41.380762 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.380701 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tx79n"] Jul 28 17:18:41.382389 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.382371 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8"] Jul 28 17:18:41.387394 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:41.387337 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a20f6b5_d596_481d_8de8_edb57100e326.slice/crio-024aff215a1ac8c0bf2e796aa919d7cd6b0595b58275662a5527b1d49bf0eeb7 WatchSource:0}: Error finding container 024aff215a1ac8c0bf2e796aa919d7cd6b0595b58275662a5527b1d49bf0eeb7: Status 404 returned error can't find the container with id 024aff215a1ac8c0bf2e796aa919d7cd6b0595b58275662a5527b1d49bf0eeb7 Jul 28 17:18:41.426491 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.426450 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h8dc4"] Jul 28 17:18:41.429914 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.429899 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.433050 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.433031 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 17:18:41.434114 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.433290 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v97v9\"" Jul 28 17:18:41.434114 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.433571 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 17:18:41.435149 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.434961 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 17:18:41.469535 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.469387 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.469535 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.469478 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.469535 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.469516 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.469726 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.469556 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nppb\" (UniqueName: \"kubernetes.io/projected/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-kube-api-access-9nppb\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.543625 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.543583 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tx79n" event={"ID":"0a20f6b5-d596-481d-8de8-edb57100e326","Type":"ContainerStarted","Data":"024aff215a1ac8c0bf2e796aa919d7cd6b0595b58275662a5527b1d49bf0eeb7"} Jul 28 17:18:41.570305 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570277 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.570472 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570318 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-sys\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.570472 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570347 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.570472 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570376 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-root\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.570472 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570402 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nppb\" (UniqueName: \"kubernetes.io/projected/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-kube-api-access-9nppb\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.570472 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570432 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-tls\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.570472 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570454 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x68zg\" (UniqueName: \"kubernetes.io/projected/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-kube-api-access-x68zg\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.570777 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570506 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-metrics-client-ca\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.570777 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570535 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.570777 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570584 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-wtmp\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.570777 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.570777 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-textfile\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.570777 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.570695 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.571068 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:41.570842 2576 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 28 17:18:41.571068 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:41.570910 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-openshift-state-metrics-tls podName:e6d0450c-1d9e-4553-9b0c-8541e66e47a9 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:42.070891328 +0000 UTC m=+99.443439907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-swjs8" (UID: "e6d0450c-1d9e-4553-9b0c-8541e66e47a9") : secret "openshift-state-metrics-tls" not found Jul 28 17:18:41.571665 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.571638 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.573220 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.573198 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.581983 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.581961 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nppb\" (UniqueName: \"kubernetes.io/projected/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-kube-api-access-9nppb\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:41.672030 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.671984 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-tls\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.672030 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.672031 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x68zg\" (UniqueName: \"kubernetes.io/projected/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-kube-api-access-x68zg\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.672269 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:41.672158 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 28 17:18:41.672269 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:41.672228 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-tls podName:d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:42.172207732 +0000 UTC m=+99.544756322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-tls") pod "node-exporter-h8dc4" (UID: "d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3") : secret "node-exporter-tls" not found Jul 28 17:18:41.672395 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.672354 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-metrics-client-ca\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.672449 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.672394 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.672505 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.672450 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-wtmp\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.672551 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.672531 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-textfile\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.672598 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.672572 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.672649 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.672607 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-sys\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.672698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.672673 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-root\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.673674 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.672759 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-root\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.673674 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.673149 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-metrics-client-ca\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.673674 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.673315 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.673674 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.673426 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-textfile\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.673674 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.673444 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-wtmp\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.673674 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.673501 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-sys\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.676441 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.676419 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:41.684162 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:41.684141 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x68zg\" (UniqueName: \"kubernetes.io/projected/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-kube-api-access-x68zg\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:42.076486 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:42.076399 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:42.079509 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:42.079484 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d0450c-1d9e-4553-9b0c-8541e66e47a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-swjs8\" (UID: \"e6d0450c-1d9e-4553-9b0c-8541e66e47a9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:42.177054 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:42.177017 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-tls\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:42.179842 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:42.179778 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3-node-exporter-tls\") pod \"node-exporter-h8dc4\" (UID: \"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3\") " pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:42.277864 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:42.277827 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" Jul 28 17:18:42.344401 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:42.344367 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h8dc4" Jul 28 17:18:42.363914 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:42.363867 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52ec06f_e4b6_47a3_8cf7_4acffc56b4a3.slice/crio-4384d4893f3605a847bf03cca89182e7890ce33b54434aa0437f3ce36331e725 WatchSource:0}: Error finding container 4384d4893f3605a847bf03cca89182e7890ce33b54434aa0437f3ce36331e725: Status 404 returned error can't find the container with id 4384d4893f3605a847bf03cca89182e7890ce33b54434aa0437f3ce36331e725 Jul 28 17:18:42.442171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:42.442132 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8"] Jul 28 17:18:42.445852 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:42.445820 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d0450c_1d9e_4553_9b0c_8541e66e47a9.slice/crio-7b2876fae1b8dc41c59923e3b1e3b9d5ef6b7d270c5d1b7f0170cd71f7197cb0 WatchSource:0}: Error finding container 7b2876fae1b8dc41c59923e3b1e3b9d5ef6b7d270c5d1b7f0170cd71f7197cb0: Status 404 returned error can't find the container with id 7b2876fae1b8dc41c59923e3b1e3b9d5ef6b7d270c5d1b7f0170cd71f7197cb0 Jul 28 17:18:42.552848 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:42.552798 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8dc4" event={"ID":"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3","Type":"ContainerStarted","Data":"4384d4893f3605a847bf03cca89182e7890ce33b54434aa0437f3ce36331e725"} Jul 28 17:18:42.558107 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:42.558076 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" event={"ID":"e6d0450c-1d9e-4553-9b0c-8541e66e47a9","Type":"ContainerStarted","Data":"29098608863ae1360e88c4c6d4a82f125d3fa0caf1d7835052cadc5eb575aeda"} Jul 28 17:18:42.558253 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:42.558146 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" event={"ID":"e6d0450c-1d9e-4553-9b0c-8541e66e47a9","Type":"ContainerStarted","Data":"7b2876fae1b8dc41c59923e3b1e3b9d5ef6b7d270c5d1b7f0170cd71f7197cb0"} Jul 28 17:18:43.564867 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:43.564822 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" event={"ID":"e6d0450c-1d9e-4553-9b0c-8541e66e47a9","Type":"ContainerStarted","Data":"c33075ba5f2992d4e20afb8dc328a5bdcfcfe58dbea5502270171935ec8233ab"} Jul 28 17:18:43.566924 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:43.566895 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tx79n" event={"ID":"0a20f6b5-d596-481d-8de8-edb57100e326","Type":"ContainerStarted","Data":"b799a11ca1ba35bc3650b810d39ff8f1fd42ca843584cce1f4b423567b56b89f"} Jul 28 17:18:43.567049 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:43.566949 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tx79n" event={"ID":"0a20f6b5-d596-481d-8de8-edb57100e326","Type":"ContainerStarted","Data":"22c14410d68d95dce97e32653c8fbad86ad28cc2193ef733577fea9d46e2e5d7"} Jul 28 17:18:43.567171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:43.567150 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tx79n" Jul 28 17:18:43.587462 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:43.587414 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tx79n" podStartSLOduration=65.929198171 podStartE2EDuration="1m7.587396167s" totalImagesPullingTime="1.658197996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:43.58539508 +0000 UTC m=+100.957943690" watchObservedRunningTime="2026-07-28 17:18:43.587396167 +0000 UTC m=+100.959944762" Jul 28 17:18:44.573860 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:44.573822 2576 generic.go:356] "Generic (PLEG): container finished" podID="d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3" containerID="84a171abca1b60a2dc650daa88e7e09f8e386de70f2bd6dd36abf3004747ecb7" exitCode=0 Jul 28 17:18:44.574329 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:44.573918 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8dc4" event={"ID":"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3","Type":"ContainerDied","Data":"84a171abca1b60a2dc650daa88e7e09f8e386de70f2bd6dd36abf3004747ecb7"} Jul 28 17:18:44.576116 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:44.576091 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" event={"ID":"e6d0450c-1d9e-4553-9b0c-8541e66e47a9","Type":"ContainerStarted","Data":"75dac2a037a48e021dcfdfd32aa350e2e6fd93bba2483b914db7080280da629e"} Jul 28 17:18:44.615590 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:44.615528 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-swjs8" podStartSLOduration=2.5150164459999997 podStartE2EDuration="3.615510528s" totalImagesPullingTime="1.100494082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:44.614184962 +0000 UTC m=+101.986733558" watchObservedRunningTime="2026-07-28 17:18:44.615510528 +0000 UTC m=+101.988059125" Jul 28 17:18:45.424161 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:45.424128 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5g2mj" Jul 28 17:18:45.582410 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:45.582374 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8dc4" event={"ID":"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3","Type":"ContainerStarted","Data":"a860e370003fe888a6752f8c07f1fd0175c35ad31cec0798f89c2e1b5c5379b9"} Jul 28 17:18:45.582815 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:45.582440 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8dc4" event={"ID":"d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3","Type":"ContainerStarted","Data":"8ec67891cffe1178be63cb40922177f38a3537d3c15464109ce3c94396769d12"} Jul 28 17:18:45.604803 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:45.604735 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h8dc4" podStartSLOduration=3.255590988 podStartE2EDuration="4.604718881s" totalImagesPullingTime="1.272033796s" totalInitContainerRuntime="77.094097ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:45.603224358 +0000 UTC m=+102.975772953" watchObservedRunningTime="2026-07-28 17:18:45.604718881 +0000 UTC m=+102.977267477" Jul 28 17:18:46.120751 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.120717 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-m552v"] Jul 28 17:18:46.152248 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.152209 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-m552v"] Jul 28 17:18:46.152440 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.152335 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" Jul 28 17:18:46.154717 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.154694 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 28 17:18:46.154860 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.154704 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dgh79\"" Jul 28 17:18:46.314215 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.314185 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d216a33-5710-4428-b12b-f5e5b8d88e8e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-m552v\" (UID: \"8d216a33-5710-4428-b12b-f5e5b8d88e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" Jul 28 17:18:46.314389 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.314295 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpcrm\" (UniqueName: \"kubernetes.io/projected/8d216a33-5710-4428-b12b-f5e5b8d88e8e-kube-api-access-rpcrm\") pod \"monitoring-plugin-68b7fc9574-m552v\" (UID: \"8d216a33-5710-4428-b12b-f5e5b8d88e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" Jul 28 17:18:46.415592 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.415503 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d216a33-5710-4428-b12b-f5e5b8d88e8e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-m552v\" (UID: \"8d216a33-5710-4428-b12b-f5e5b8d88e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" Jul 28 17:18:46.415747 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.415636 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpcrm\" (UniqueName: \"kubernetes.io/projected/8d216a33-5710-4428-b12b-f5e5b8d88e8e-kube-api-access-rpcrm\") pod \"monitoring-plugin-68b7fc9574-m552v\" (UID: \"8d216a33-5710-4428-b12b-f5e5b8d88e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" Jul 28 17:18:46.415747 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:46.415656 2576 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 28 17:18:46.415747 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:18:46.415722 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d216a33-5710-4428-b12b-f5e5b8d88e8e-monitoring-plugin-cert podName:8d216a33-5710-4428-b12b-f5e5b8d88e8e nodeName:}" failed. No retries permitted until 2026-07-28 17:18:46.915700507 +0000 UTC m=+104.288249080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/8d216a33-5710-4428-b12b-f5e5b8d88e8e-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-m552v" (UID: "8d216a33-5710-4428-b12b-f5e5b8d88e8e") : secret "monitoring-plugin-cert" not found Jul 28 17:18:46.424985 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.424963 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpcrm\" (UniqueName: \"kubernetes.io/projected/8d216a33-5710-4428-b12b-f5e5b8d88e8e-kube-api-access-rpcrm\") pod \"monitoring-plugin-68b7fc9574-m552v\" (UID: \"8d216a33-5710-4428-b12b-f5e5b8d88e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" Jul 28 17:18:46.919160 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.919126 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d216a33-5710-4428-b12b-f5e5b8d88e8e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-m552v\" (UID: \"8d216a33-5710-4428-b12b-f5e5b8d88e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" Jul 28 17:18:46.921436 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:46.921408 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d216a33-5710-4428-b12b-f5e5b8d88e8e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-m552v\" (UID: \"8d216a33-5710-4428-b12b-f5e5b8d88e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" Jul 28 17:18:47.061916 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:47.061876 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" Jul 28 17:18:47.206330 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:47.206259 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-m552v"] Jul 28 17:18:47.209147 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:47.209119 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d216a33_5710_4428_b12b_f5e5b8d88e8e.slice/crio-665e182532c279a63f4f06bbd20fd90374b3933f40c07a050a9d2d1b7ac40ab8 WatchSource:0}: Error finding container 665e182532c279a63f4f06bbd20fd90374b3933f40c07a050a9d2d1b7ac40ab8: Status 404 returned error can't find the container with id 665e182532c279a63f4f06bbd20fd90374b3933f40c07a050a9d2d1b7ac40ab8 Jul 28 17:18:47.596326 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:47.596283 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" event={"ID":"8d216a33-5710-4428-b12b-f5e5b8d88e8e","Type":"ContainerStarted","Data":"665e182532c279a63f4f06bbd20fd90374b3933f40c07a050a9d2d1b7ac40ab8"} Jul 28 17:18:49.604647 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:49.604610 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" event={"ID":"8d216a33-5710-4428-b12b-f5e5b8d88e8e","Type":"ContainerStarted","Data":"f543d54fb87f60e380d6aa9d7f5fc1168be359e4e3e332ce19c4095c73791928"} Jul 28 17:18:49.605060 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:49.604773 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" Jul 28 17:18:49.609418 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:49.609397 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" Jul 28 17:18:49.622153 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:49.622112 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-m552v" podStartSLOduration=1.902900365 podStartE2EDuration="3.622097615s" totalImagesPullingTime="1.71919725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:49.621410356 +0000 UTC m=+106.993958953" watchObservedRunningTime="2026-07-28 17:18:49.622097615 +0000 UTC m=+106.994646210" Jul 28 17:18:52.185429 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.185393 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d9f64c894-shhrs"] Jul 28 17:18:52.187527 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.187501 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.190145 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.190118 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 17:18:52.190274 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.190181 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 17:18:52.190438 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.190414 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 17:18:52.191255 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.191236 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-66zv9\"" Jul 28 17:18:52.191478 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.191461 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 17:18:52.191635 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.191471 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 17:18:52.191635 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.191595 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 17:18:52.191635 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.191631 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 17:18:52.199800 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.196720 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 17:18:52.201841 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.201819 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d9f64c894-shhrs"] Jul 28 17:18:52.363980 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.363940 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcz6l\" (UniqueName: \"kubernetes.io/projected/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-kube-api-access-hcz6l\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.364133 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.363989 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-service-ca\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.364133 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.364051 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-config\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.364218 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.364137 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-oauth-config\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.364252 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.364219 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-trusted-ca-bundle\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.364252 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.364241 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-oauth-serving-cert\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.364337 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.364264 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-serving-cert\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.465643 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.465554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-oauth-config\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.465835 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.465646 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-trusted-ca-bundle\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.465835 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.465670 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-oauth-serving-cert\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.465835 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.465690 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-serving-cert\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.465835 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.465723 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcz6l\" (UniqueName: \"kubernetes.io/projected/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-kube-api-access-hcz6l\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.465835 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.465778 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-service-ca\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.466091 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.465863 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-config\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.466646 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.466623 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-config\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.466747 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.466644 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-oauth-serving-cert\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.466747 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.466644 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-service-ca\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.466747 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.466717 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-trusted-ca-bundle\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.468031 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.468011 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-oauth-config\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.468185 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.468164 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-serving-cert\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.474492 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.474463 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcz6l\" (UniqueName: \"kubernetes.io/projected/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-kube-api-access-hcz6l\") pod \"console-5d9f64c894-shhrs\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.497297 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.497269 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:18:52.625596 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:52.625565 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d9f64c894-shhrs"] Jul 28 17:18:52.628681 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:18:52.628646 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd9419c0_f15b_4f3a_afd0_bbfb586aa24f.slice/crio-569f80ce72820677daa1624bef57ec149e8a5b430807d30c27caa2c1bd7a5655 WatchSource:0}: Error finding container 569f80ce72820677daa1624bef57ec149e8a5b430807d30c27caa2c1bd7a5655: Status 404 returned error can't find the container with id 569f80ce72820677daa1624bef57ec149e8a5b430807d30c27caa2c1bd7a5655 Jul 28 17:18:53.578557 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:53.578525 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tx79n" Jul 28 17:18:53.626847 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:53.626736 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9f64c894-shhrs" event={"ID":"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f","Type":"ContainerStarted","Data":"569f80ce72820677daa1624bef57ec149e8a5b430807d30c27caa2c1bd7a5655"} Jul 28 17:18:54.516998 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:54.516970 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b7f4fb6c9-l874l" Jul 28 17:18:55.635467 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:55.635387 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9f64c894-shhrs" event={"ID":"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f","Type":"ContainerStarted","Data":"cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b"} Jul 28 17:18:55.656308 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:18:55.656255 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d9f64c894-shhrs" podStartSLOduration=1.02202694 podStartE2EDuration="3.656237139s" totalImagesPullingTime="2.634210199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:55.654453883 +0000 UTC m=+113.027002481" watchObservedRunningTime="2026-07-28 17:18:55.656237139 +0000 UTC m=+113.028785737" Jul 28 17:19:02.497967 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:02.497925 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:19:02.498389 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:02.497985 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:19:02.502706 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:02.502682 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:19:02.664177 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:02.664147 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:19:12.934232 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:12.934195 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:19:12.936379 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:12.936358 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e761eac4-98d4-494a-be69-82b2f0e6e4d4-metrics-certs\") pod \"network-metrics-daemon-s9qjh\" (UID: \"e761eac4-98d4-494a-be69-82b2f0e6e4d4\") " pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:19:12.967623 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:12.967596 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvwzr\"" Jul 28 17:19:12.975835 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:12.975820 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s9qjh" Jul 28 17:19:13.501840 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:13.501814 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s9qjh"] Jul 28 17:19:13.504386 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:19:13.504355 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode761eac4_98d4_494a_be69_82b2f0e6e4d4.slice/crio-e99e63d16a8ce493419953162f946ce4823aa246b49a3bb3d5cd0ab5ea43df9d WatchSource:0}: Error finding container e99e63d16a8ce493419953162f946ce4823aa246b49a3bb3d5cd0ab5ea43df9d: Status 404 returned error can't find the container with id e99e63d16a8ce493419953162f946ce4823aa246b49a3bb3d5cd0ab5ea43df9d Jul 28 17:19:13.697489 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:13.697406 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s9qjh" event={"ID":"e761eac4-98d4-494a-be69-82b2f0e6e4d4","Type":"ContainerStarted","Data":"e99e63d16a8ce493419953162f946ce4823aa246b49a3bb3d5cd0ab5ea43df9d"} Jul 28 17:19:14.703360 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:14.703323 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s9qjh" event={"ID":"e761eac4-98d4-494a-be69-82b2f0e6e4d4","Type":"ContainerStarted","Data":"22e47f3afe195c6f440f624006e46066553ca5f38b3bb9dfc2e3f83decaba689"} Jul 28 17:19:14.703710 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:14.703389 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s9qjh" event={"ID":"e761eac4-98d4-494a-be69-82b2f0e6e4d4","Type":"ContainerStarted","Data":"4b709be781fa4ac396c8a627741baa1abdb53f29ae34e6e0b5010a5d95246c9f"} Jul 28 17:19:14.723819 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:14.722474 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s9qjh" podStartSLOduration=130.700397963 podStartE2EDuration="2m11.722452251s" totalImagesPullingTime="1.022054288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:19:14.719696019 +0000 UTC m=+132.092244617" watchObservedRunningTime="2026-07-28 17:19:14.722452251 +0000 UTC m=+132.095000848" Jul 28 17:19:19.721759 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:19.721681 2576 generic.go:356] "Generic (PLEG): container finished" podID="eef7416b-d05b-4b67-b553-dc593460efb1" containerID="ef25cfd46f19147a8b6b6caec06a9326eefc72a8d3b1cc2af6e05e79cd68c427" exitCode=0 Jul 28 17:19:19.722279 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:19.721762 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fdb6z" event={"ID":"eef7416b-d05b-4b67-b553-dc593460efb1","Type":"ContainerDied","Data":"ef25cfd46f19147a8b6b6caec06a9326eefc72a8d3b1cc2af6e05e79cd68c427"} Jul 28 17:19:19.722279 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:19.722185 2576 scope.go:117] "RemoveContainer" containerID="ef25cfd46f19147a8b6b6caec06a9326eefc72a8d3b1cc2af6e05e79cd68c427" Jul 28 17:19:20.727399 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:19:20.727367 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-fdb6z" event={"ID":"eef7416b-d05b-4b67-b553-dc593460efb1","Type":"ContainerStarted","Data":"c4f3270bd2fa1cf6b815cbacf5a919c2d84686c8d31c5b00d94f2d35f83ad213"} Jul 28 17:20:35.257836 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:20:35.257797 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d9f64c894-shhrs"] Jul 28 17:21:00.281682 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.281623 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d9f64c894-shhrs" podUID="fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" containerName="console" containerID="cri-o://cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b" gracePeriod=15 Jul 28 17:21:00.529102 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.529072 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:21:00.575922 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.575841 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-serving-cert\") pod \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " Jul 28 17:21:00.575922 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.575910 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-config\") pod \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " Jul 28 17:21:00.576174 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.575942 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-oauth-config\") pod \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " Jul 28 17:21:00.576174 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.575968 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-trusted-ca-bundle\") pod \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " Jul 28 17:21:00.576174 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.576018 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-service-ca\") pod \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " Jul 28 17:21:00.576174 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.576052 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcz6l\" (UniqueName: \"kubernetes.io/projected/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-kube-api-access-hcz6l\") pod \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " Jul 28 17:21:00.576174 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.576084 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-oauth-serving-cert\") pod \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\" (UID: \"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f\") " Jul 28 17:21:00.576521 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.576384 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-config" (OuterVolumeSpecName: "console-config") pod "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" (UID: "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:21:00.576611 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.576577 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" (UID: "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:21:00.576611 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.576589 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-service-ca" (OuterVolumeSpecName: "service-ca") pod "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" (UID: "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:21:00.576711 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.576635 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" (UID: "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:21:00.578221 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.578194 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" (UID: "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:21:00.578310 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.578247 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-kube-api-access-hcz6l" (OuterVolumeSpecName: "kube-api-access-hcz6l") pod "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" (UID: "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f"). InnerVolumeSpecName "kube-api-access-hcz6l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:21:00.578310 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.578275 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" (UID: "fd9419c0-f15b-4f3a-afd0-bbfb586aa24f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:21:00.676718 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.676663 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:21:00.676718 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.676710 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-oauth-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:21:00.676718 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.676725 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-trusted-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:21:00.676992 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.676740 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-service-ca\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:21:00.676992 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.676753 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcz6l\" (UniqueName: \"kubernetes.io/projected/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-kube-api-access-hcz6l\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:21:00.676992 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.676765 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-oauth-serving-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:21:00.676992 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:00.676776 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f-console-serving-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:21:01.089412 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:01.089373 2576 generic.go:356] "Generic (PLEG): container finished" podID="fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" containerID="cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b" exitCode=0 Jul 28 17:21:01.089556 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:01.089428 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9f64c894-shhrs" Jul 28 17:21:01.089556 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:01.089437 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9f64c894-shhrs" event={"ID":"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f","Type":"ContainerDied","Data":"cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b"} Jul 28 17:21:01.089556 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:01.089516 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9f64c894-shhrs" event={"ID":"fd9419c0-f15b-4f3a-afd0-bbfb586aa24f","Type":"ContainerDied","Data":"569f80ce72820677daa1624bef57ec149e8a5b430807d30c27caa2c1bd7a5655"} Jul 28 17:21:01.089556 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:01.089543 2576 scope.go:117] "RemoveContainer" containerID="cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b" Jul 28 17:21:01.097750 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:01.097732 2576 scope.go:117] "RemoveContainer" containerID="cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b" Jul 28 17:21:01.098034 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:21:01.098013 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b\": container with ID starting with cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b not found: ID does not exist" containerID="cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b" Jul 28 17:21:01.098107 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:01.098040 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b"} err="failed to get container status \"cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b\": rpc error: code = NotFound desc = could not find container \"cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b\": container with ID starting with cca066de510e0de553bcdb0334a0de9e29bc01901a941215ea5ca27609f4fd2b not found: ID does not exist" Jul 28 17:21:01.111576 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:01.111550 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d9f64c894-shhrs"] Jul 28 17:21:01.116059 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:01.116037 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d9f64c894-shhrs"] Jul 28 17:21:01.142923 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:01.142901 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" path="/var/lib/kubelet/pods/fd9419c0-f15b-4f3a-afd0-bbfb586aa24f/volumes" Jul 28 17:21:02.048902 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.048866 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv"] Jul 28 17:21:02.049309 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.049132 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" containerName="console" Jul 28 17:21:02.049309 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.049142 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" containerName="console" Jul 28 17:21:02.049309 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.049195 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd9419c0-f15b-4f3a-afd0-bbfb586aa24f" containerName="console" Jul 28 17:21:02.053241 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.053222 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv" Jul 28 17:21:02.055904 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.055883 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 28 17:21:02.056030 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.055919 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 28 17:21:02.057341 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.057325 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 28 17:21:02.057460 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.057375 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-sxxsf\"" Jul 28 17:21:02.057460 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.057427 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 28 17:21:02.063981 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.063960 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv"] Jul 28 17:21:02.086973 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.086950 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bhvv\" (UniqueName: \"kubernetes.io/projected/0f554df7-b78f-43cd-8da2-bdedd31565f1-kube-api-access-8bhvv\") pod \"managed-serviceaccount-addon-agent-547b69d59d-bplxv\" (UID: \"0f554df7-b78f-43cd-8da2-bdedd31565f1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv" Jul 28 17:21:02.087075 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.086998 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0f554df7-b78f-43cd-8da2-bdedd31565f1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-547b69d59d-bplxv\" (UID: \"0f554df7-b78f-43cd-8da2-bdedd31565f1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv" Jul 28 17:21:02.157508 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.157480 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9"] Jul 28 17:21:02.160532 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.160512 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:02.162992 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.162975 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 28 17:21:02.169605 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.169582 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9"] Jul 28 17:21:02.187702 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.187676 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bhvv\" (UniqueName: \"kubernetes.io/projected/0f554df7-b78f-43cd-8da2-bdedd31565f1-kube-api-access-8bhvv\") pod \"managed-serviceaccount-addon-agent-547b69d59d-bplxv\" (UID: \"0f554df7-b78f-43cd-8da2-bdedd31565f1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv" Jul 28 17:21:02.187871 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.187729 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0f554df7-b78f-43cd-8da2-bdedd31565f1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-547b69d59d-bplxv\" (UID: \"0f554df7-b78f-43cd-8da2-bdedd31565f1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv" Jul 28 17:21:02.190324 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.190298 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0f554df7-b78f-43cd-8da2-bdedd31565f1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-547b69d59d-bplxv\" (UID: \"0f554df7-b78f-43cd-8da2-bdedd31565f1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv" Jul 28 17:21:02.195510 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.195488 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bhvv\" (UniqueName: \"kubernetes.io/projected/0f554df7-b78f-43cd-8da2-bdedd31565f1-kube-api-access-8bhvv\") pod \"managed-serviceaccount-addon-agent-547b69d59d-bplxv\" (UID: \"0f554df7-b78f-43cd-8da2-bdedd31565f1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv" Jul 28 17:21:02.288373 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.288341 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9a0a0875-5cda-4a09-a9a5-2a7d617d8a68-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bcf7754bb-djng9\" (UID: \"9a0a0875-5cda-4a09-a9a5-2a7d617d8a68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:02.288504 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.288381 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a0a0875-5cda-4a09-a9a5-2a7d617d8a68-tmp\") pod \"klusterlet-addon-workmgr-5bcf7754bb-djng9\" (UID: \"9a0a0875-5cda-4a09-a9a5-2a7d617d8a68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:02.288504 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.288448 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftmz9\" (UniqueName: \"kubernetes.io/projected/9a0a0875-5cda-4a09-a9a5-2a7d617d8a68-kube-api-access-ftmz9\") pod \"klusterlet-addon-workmgr-5bcf7754bb-djng9\" (UID: \"9a0a0875-5cda-4a09-a9a5-2a7d617d8a68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:02.372635 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.372556 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv" Jul 28 17:21:02.388799 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.388762 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftmz9\" (UniqueName: \"kubernetes.io/projected/9a0a0875-5cda-4a09-a9a5-2a7d617d8a68-kube-api-access-ftmz9\") pod \"klusterlet-addon-workmgr-5bcf7754bb-djng9\" (UID: \"9a0a0875-5cda-4a09-a9a5-2a7d617d8a68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:02.388901 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.388854 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9a0a0875-5cda-4a09-a9a5-2a7d617d8a68-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bcf7754bb-djng9\" (UID: \"9a0a0875-5cda-4a09-a9a5-2a7d617d8a68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:02.388901 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.388893 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a0a0875-5cda-4a09-a9a5-2a7d617d8a68-tmp\") pod \"klusterlet-addon-workmgr-5bcf7754bb-djng9\" (UID: \"9a0a0875-5cda-4a09-a9a5-2a7d617d8a68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:02.389296 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.389275 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a0a0875-5cda-4a09-a9a5-2a7d617d8a68-tmp\") pod \"klusterlet-addon-workmgr-5bcf7754bb-djng9\" (UID: \"9a0a0875-5cda-4a09-a9a5-2a7d617d8a68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:02.391380 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.391355 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9a0a0875-5cda-4a09-a9a5-2a7d617d8a68-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bcf7754bb-djng9\" (UID: \"9a0a0875-5cda-4a09-a9a5-2a7d617d8a68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:02.406255 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.406229 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftmz9\" (UniqueName: \"kubernetes.io/projected/9a0a0875-5cda-4a09-a9a5-2a7d617d8a68-kube-api-access-ftmz9\") pod \"klusterlet-addon-workmgr-5bcf7754bb-djng9\" (UID: \"9a0a0875-5cda-4a09-a9a5-2a7d617d8a68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:02.469269 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.469239 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:02.502159 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.502002 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv"] Jul 28 17:21:02.504879 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:21:02.504849 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f554df7_b78f_43cd_8da2_bdedd31565f1.slice/crio-bd7a6ee97628fdf302ad50c2e6a9c0bb4e192e54719a8b3401a6349db471b4a6 WatchSource:0}: Error finding container bd7a6ee97628fdf302ad50c2e6a9c0bb4e192e54719a8b3401a6349db471b4a6: Status 404 returned error can't find the container with id bd7a6ee97628fdf302ad50c2e6a9c0bb4e192e54719a8b3401a6349db471b4a6 Jul 28 17:21:02.594602 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:02.594579 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9"] Jul 28 17:21:02.596960 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:21:02.596936 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a0a0875_5cda_4a09_a9a5_2a7d617d8a68.slice/crio-e7e26cd598277ce722b6069f697d2083145c709c620b87d9b1216fb271d9bda2 WatchSource:0}: Error finding container e7e26cd598277ce722b6069f697d2083145c709c620b87d9b1216fb271d9bda2: Status 404 returned error can't find the container with id e7e26cd598277ce722b6069f697d2083145c709c620b87d9b1216fb271d9bda2 Jul 28 17:21:03.098525 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:03.098488 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv" event={"ID":"0f554df7-b78f-43cd-8da2-bdedd31565f1","Type":"ContainerStarted","Data":"bd7a6ee97628fdf302ad50c2e6a9c0bb4e192e54719a8b3401a6349db471b4a6"} Jul 28 17:21:03.099507 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:03.099485 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" event={"ID":"9a0a0875-5cda-4a09-a9a5-2a7d617d8a68","Type":"ContainerStarted","Data":"e7e26cd598277ce722b6069f697d2083145c709c620b87d9b1216fb271d9bda2"} Jul 28 17:21:07.120588 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:07.120543 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv" event={"ID":"0f554df7-b78f-43cd-8da2-bdedd31565f1","Type":"ContainerStarted","Data":"7435cacd1351bdf8d1352f754a595ac572d913ba86636438d2c5da1c88648b6a"} Jul 28 17:21:07.122088 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:07.122064 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" event={"ID":"9a0a0875-5cda-4a09-a9a5-2a7d617d8a68","Type":"ContainerStarted","Data":"69d35554f874d10b528e389515b541bbef900c7604fd5e686ac7418717295571"} Jul 28 17:21:07.122236 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:07.122139 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:07.123874 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:07.123851 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" Jul 28 17:21:07.139352 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:07.139306 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-547b69d59d-bplxv" podStartSLOduration=0.673564885 podStartE2EDuration="5.139288843s" totalImagesPullingTime="4.465723958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:21:07.13841826 +0000 UTC m=+244.510966857" watchObservedRunningTime="2026-07-28 17:21:07.139288843 +0000 UTC m=+244.511837440" Jul 28 17:21:07.157861 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:07.157818 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bcf7754bb-djng9" podStartSLOduration=0.773082904 podStartE2EDuration="5.157803604s" totalImagesPullingTime="4.3847207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:21:07.157337258 +0000 UTC m=+244.529885855" watchObservedRunningTime="2026-07-28 17:21:07.157803604 +0000 UTC m=+244.530352191" Jul 28 17:21:13.529142 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.529105 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n"] Jul 28 17:21:13.532585 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.532566 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:13.535162 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.535141 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 28 17:21:13.535284 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.535141 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 28 17:21:13.536317 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.536298 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b2jgt\"" Jul 28 17:21:13.541356 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.541331 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n"] Jul 28 17:21:13.560539 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.560509 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zjj2\" (UniqueName: \"kubernetes.io/projected/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-kube-api-access-6zjj2\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:13.560660 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.560549 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:13.560660 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.560568 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:13.660895 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.660860 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:13.661018 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.660944 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zjj2\" (UniqueName: \"kubernetes.io/projected/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-kube-api-access-6zjj2\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:13.661018 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.660979 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:13.661237 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.661217 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:13.661288 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.661268 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:13.670641 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.670615 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zjj2\" (UniqueName: \"kubernetes.io/projected/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-kube-api-access-6zjj2\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:13.842076 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.842025 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:13.967997 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:13.967966 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n"] Jul 28 17:21:13.971131 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:21:13.971103 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c2c2c9_4e6e_4a34_ad48_0cd0fa978a50.slice/crio-9b24de6044bdc4785269990975f29b72dcb6b817f994acfae4ea469558dcdbc1 WatchSource:0}: Error finding container 9b24de6044bdc4785269990975f29b72dcb6b817f994acfae4ea469558dcdbc1: Status 404 returned error can't find the container with id 9b24de6044bdc4785269990975f29b72dcb6b817f994acfae4ea469558dcdbc1 Jul 28 17:21:14.146473 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:14.146399 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" event={"ID":"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50","Type":"ContainerStarted","Data":"9b24de6044bdc4785269990975f29b72dcb6b817f994acfae4ea469558dcdbc1"} Jul 28 17:21:21.176091 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:21.176063 2576 generic.go:356] "Generic (PLEG): container finished" podID="96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" containerID="cac3cebcbf63c11fc9e191e8baa366422e6b2460c8edf36c4a9b5bcff8e6b4ce" exitCode=0 Jul 28 17:21:21.176403 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:21.176122 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" event={"ID":"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50","Type":"ContainerDied","Data":"cac3cebcbf63c11fc9e191e8baa366422e6b2460c8edf36c4a9b5bcff8e6b4ce"} Jul 28 17:21:23.185296 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:23.185268 2576 generic.go:356] "Generic (PLEG): container finished" podID="96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" containerID="c0a56026f95c450bb20ab6416ad4a5a545c84f3c23dff782c544477ab419d73e" exitCode=0 Jul 28 17:21:23.185611 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:23.185322 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" event={"ID":"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50","Type":"ContainerDied","Data":"c0a56026f95c450bb20ab6416ad4a5a545c84f3c23dff782c544477ab419d73e"} Jul 28 17:21:30.217770 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:30.217679 2576 generic.go:356] "Generic (PLEG): container finished" podID="96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" containerID="7ff17dccb204493b0f67d537baf3724443d47abcccabfb3a954e7260ebe236dd" exitCode=0 Jul 28 17:21:30.218137 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:30.217761 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" event={"ID":"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50","Type":"ContainerDied","Data":"7ff17dccb204493b0f67d537baf3724443d47abcccabfb3a954e7260ebe236dd"} Jul 28 17:21:31.346776 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:31.346753 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:31.413062 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:31.413030 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zjj2\" (UniqueName: \"kubernetes.io/projected/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-kube-api-access-6zjj2\") pod \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " Jul 28 17:21:31.413193 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:31.413076 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-bundle\") pod \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " Jul 28 17:21:31.413193 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:31.413117 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-util\") pod \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\" (UID: \"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50\") " Jul 28 17:21:31.413750 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:31.413720 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-bundle" (OuterVolumeSpecName: "bundle") pod "96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" (UID: "96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:21:31.415134 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:31.415101 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-kube-api-access-6zjj2" (OuterVolumeSpecName: "kube-api-access-6zjj2") pod "96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" (UID: "96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50"). InnerVolumeSpecName "kube-api-access-6zjj2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:21:31.417655 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:31.417634 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-util" (OuterVolumeSpecName: "util") pod "96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" (UID: "96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:21:31.513626 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:31.513603 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-util\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:21:31.513744 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:31.513626 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zjj2\" (UniqueName: \"kubernetes.io/projected/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-kube-api-access-6zjj2\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:21:31.513744 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:31.513638 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:21:32.230242 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:32.230212 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" event={"ID":"96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50","Type":"ContainerDied","Data":"9b24de6044bdc4785269990975f29b72dcb6b817f994acfae4ea469558dcdbc1"} Jul 28 17:21:32.230401 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:32.230261 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b24de6044bdc4785269990975f29b72dcb6b817f994acfae4ea469558dcdbc1" Jul 28 17:21:32.230401 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:32.230222 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5nbv9n" Jul 28 17:21:36.246569 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.246532 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld"] Jul 28 17:21:36.246988 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.246818 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" containerName="extract" Jul 28 17:21:36.246988 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.246830 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" containerName="extract" Jul 28 17:21:36.246988 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.246842 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" containerName="util" Jul 28 17:21:36.246988 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.246847 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" containerName="util" Jul 28 17:21:36.246988 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.246862 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" containerName="pull" Jul 28 17:21:36.246988 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.246868 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" containerName="pull" Jul 28 17:21:36.246988 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.246910 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c2c2c9-4e6e-4a34-ad48-0cd0fa978a50" containerName="extract" Jul 28 17:21:36.251373 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.251355 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" Jul 28 17:21:36.255504 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.255482 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 28 17:21:36.255773 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.255751 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 28 17:21:36.255854 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.255757 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 28 17:21:36.255854 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.255759 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-bl9xb\"" Jul 28 17:21:36.267247 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.267220 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld"] Jul 28 17:21:36.351912 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.351874 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/22311460-5d16-41da-b131-e5405edca56f-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mwlld\" (UID: \"22311460-5d16-41da-b131-e5405edca56f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" Jul 28 17:21:36.351912 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.351919 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kpw5\" (UniqueName: \"kubernetes.io/projected/22311460-5d16-41da-b131-e5405edca56f-kube-api-access-5kpw5\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mwlld\" (UID: \"22311460-5d16-41da-b131-e5405edca56f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" Jul 28 17:21:36.452905 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.452867 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/22311460-5d16-41da-b131-e5405edca56f-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mwlld\" (UID: \"22311460-5d16-41da-b131-e5405edca56f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" Jul 28 17:21:36.453085 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.452918 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kpw5\" (UniqueName: \"kubernetes.io/projected/22311460-5d16-41da-b131-e5405edca56f-kube-api-access-5kpw5\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mwlld\" (UID: \"22311460-5d16-41da-b131-e5405edca56f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" Jul 28 17:21:36.455205 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.455178 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/22311460-5d16-41da-b131-e5405edca56f-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mwlld\" (UID: \"22311460-5d16-41da-b131-e5405edca56f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" Jul 28 17:21:36.462525 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.462498 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kpw5\" (UniqueName: \"kubernetes.io/projected/22311460-5d16-41da-b131-e5405edca56f-kube-api-access-5kpw5\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mwlld\" (UID: \"22311460-5d16-41da-b131-e5405edca56f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" Jul 28 17:21:36.561269 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.561233 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" Jul 28 17:21:36.715502 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:36.715469 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld"] Jul 28 17:21:36.718760 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:21:36.718734 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22311460_5d16_41da_b131_e5405edca56f.slice/crio-f9ee1227acf1936a44b86799917b0b0af831770c0406496405e523ae7957d9c4 WatchSource:0}: Error finding container f9ee1227acf1936a44b86799917b0b0af831770c0406496405e523ae7957d9c4: Status 404 returned error can't find the container with id f9ee1227acf1936a44b86799917b0b0af831770c0406496405e523ae7957d9c4 Jul 28 17:21:37.251033 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:37.250993 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" event={"ID":"22311460-5d16-41da-b131-e5405edca56f","Type":"ContainerStarted","Data":"f9ee1227acf1936a44b86799917b0b0af831770c0406496405e523ae7957d9c4"} Jul 28 17:21:41.272976 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:41.272934 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" event={"ID":"22311460-5d16-41da-b131-e5405edca56f","Type":"ContainerStarted","Data":"3cbfee7669a65bf1b5f4d3f0de90f485e46cfa97eddde98977bff329192fc756"} Jul 28 17:21:41.273447 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:41.273030 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" Jul 28 17:21:41.302742 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:41.302679 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" podStartSLOduration=1.4678525279999999 podStartE2EDuration="5.302659804s" totalImagesPullingTime="3.834807276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:21:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:21:41.302248535 +0000 UTC m=+278.674797132" watchObservedRunningTime="2026-07-28 17:21:41.302659804 +0000 UTC m=+278.675208402" Jul 28 17:21:42.664385 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.664344 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-9fkwm"] Jul 28 17:21:42.680698 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.680662 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-9fkwm"] Jul 28 17:21:42.680865 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.680767 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" Jul 28 17:21:42.684366 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.684346 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-lcrgr\"" Jul 28 17:21:42.684366 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.684357 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 28 17:21:42.684550 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.684391 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 28 17:21:42.805573 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.805543 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/546eea9a-543d-45ac-a8c4-5544dd2fd46c-certificates\") pod \"keda-admission-56f9798bb4-9fkwm\" (UID: \"546eea9a-543d-45ac-a8c4-5544dd2fd46c\") " pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" Jul 28 17:21:42.805729 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.805581 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgltt\" (UniqueName: \"kubernetes.io/projected/546eea9a-543d-45ac-a8c4-5544dd2fd46c-kube-api-access-tgltt\") pod \"keda-admission-56f9798bb4-9fkwm\" (UID: \"546eea9a-543d-45ac-a8c4-5544dd2fd46c\") " pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" Jul 28 17:21:42.906427 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.906389 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/546eea9a-543d-45ac-a8c4-5544dd2fd46c-certificates\") pod \"keda-admission-56f9798bb4-9fkwm\" (UID: \"546eea9a-543d-45ac-a8c4-5544dd2fd46c\") " pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" Jul 28 17:21:42.906427 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.906429 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgltt\" (UniqueName: \"kubernetes.io/projected/546eea9a-543d-45ac-a8c4-5544dd2fd46c-kube-api-access-tgltt\") pod \"keda-admission-56f9798bb4-9fkwm\" (UID: \"546eea9a-543d-45ac-a8c4-5544dd2fd46c\") " pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" Jul 28 17:21:42.908833 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.908780 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/546eea9a-543d-45ac-a8c4-5544dd2fd46c-certificates\") pod \"keda-admission-56f9798bb4-9fkwm\" (UID: \"546eea9a-543d-45ac-a8c4-5544dd2fd46c\") " pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" Jul 28 17:21:42.915226 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.915174 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgltt\" (UniqueName: \"kubernetes.io/projected/546eea9a-543d-45ac-a8c4-5544dd2fd46c-kube-api-access-tgltt\") pod \"keda-admission-56f9798bb4-9fkwm\" (UID: \"546eea9a-543d-45ac-a8c4-5544dd2fd46c\") " pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" Jul 28 17:21:42.993139 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:42.993104 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" Jul 28 17:21:43.144552 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:43.144515 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-9fkwm"] Jul 28 17:21:43.145003 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:21:43.144977 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod546eea9a_543d_45ac_a8c4_5544dd2fd46c.slice/crio-ae913b5d2426c9cd9c14863cc7e7fc7ae1239a15ca7cf7464508a7de71512a98 WatchSource:0}: Error finding container ae913b5d2426c9cd9c14863cc7e7fc7ae1239a15ca7cf7464508a7de71512a98: Status 404 returned error can't find the container with id ae913b5d2426c9cd9c14863cc7e7fc7ae1239a15ca7cf7464508a7de71512a98 Jul 28 17:21:43.281711 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:43.281676 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" event={"ID":"546eea9a-543d-45ac-a8c4-5544dd2fd46c","Type":"ContainerStarted","Data":"ae913b5d2426c9cd9c14863cc7e7fc7ae1239a15ca7cf7464508a7de71512a98"} Jul 28 17:21:46.296606 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:46.296564 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" event={"ID":"546eea9a-543d-45ac-a8c4-5544dd2fd46c","Type":"ContainerStarted","Data":"25e0af47b009f386baf256458f66d44271be10439e60f9df0cb6faa87e538b1d"} Jul 28 17:21:46.297071 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:46.296649 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" Jul 28 17:21:46.315011 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:21:46.314966 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" podStartSLOduration=2.195161904 podStartE2EDuration="4.314949386s" totalImagesPullingTime="2.119787482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:21:46.313959627 +0000 UTC m=+283.686508213" watchObservedRunningTime="2026-07-28 17:21:46.314949386 +0000 UTC m=+283.687497984" Jul 28 17:22:02.279831 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:02.279775 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mwlld" Jul 28 17:22:02.999272 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:02.999240 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:22:03.002979 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:03.002894 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:22:03.010876 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:03.010856 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 17:22:07.303003 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:07.302959 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-9fkwm" Jul 28 17:22:57.744606 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.744524 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-7mvnz"] Jul 28 17:22:57.747715 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.747695 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" Jul 28 17:22:57.750198 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.750161 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 28 17:22:57.750334 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.750198 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-jjxpj\"" Jul 28 17:22:57.751165 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.751146 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 28 17:22:57.751264 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.751230 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 28 17:22:57.757970 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.757948 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-7mvnz"] Jul 28 17:22:57.895496 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.895455 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8de49ea-a46e-4a86-81fb-734718e97b4e-cert\") pod \"llmisvc-controller-manager-84fc959947-7mvnz\" (UID: \"e8de49ea-a46e-4a86-81fb-734718e97b4e\") " pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" Jul 28 17:22:57.895693 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.895552 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qxkx\" (UniqueName: \"kubernetes.io/projected/e8de49ea-a46e-4a86-81fb-734718e97b4e-kube-api-access-6qxkx\") pod \"llmisvc-controller-manager-84fc959947-7mvnz\" (UID: \"e8de49ea-a46e-4a86-81fb-734718e97b4e\") " pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" Jul 28 17:22:57.996277 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.996199 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qxkx\" (UniqueName: \"kubernetes.io/projected/e8de49ea-a46e-4a86-81fb-734718e97b4e-kube-api-access-6qxkx\") pod \"llmisvc-controller-manager-84fc959947-7mvnz\" (UID: \"e8de49ea-a46e-4a86-81fb-734718e97b4e\") " pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" Jul 28 17:22:57.996277 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.996246 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8de49ea-a46e-4a86-81fb-734718e97b4e-cert\") pod \"llmisvc-controller-manager-84fc959947-7mvnz\" (UID: \"e8de49ea-a46e-4a86-81fb-734718e97b4e\") " pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" Jul 28 17:22:57.998713 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:57.998688 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8de49ea-a46e-4a86-81fb-734718e97b4e-cert\") pod \"llmisvc-controller-manager-84fc959947-7mvnz\" (UID: \"e8de49ea-a46e-4a86-81fb-734718e97b4e\") " pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" Jul 28 17:22:58.005964 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:58.005940 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qxkx\" (UniqueName: \"kubernetes.io/projected/e8de49ea-a46e-4a86-81fb-734718e97b4e-kube-api-access-6qxkx\") pod \"llmisvc-controller-manager-84fc959947-7mvnz\" (UID: \"e8de49ea-a46e-4a86-81fb-734718e97b4e\") " pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" Jul 28 17:22:58.059633 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:58.059601 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" Jul 28 17:22:58.186269 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:58.186236 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-7mvnz"] Jul 28 17:22:58.190239 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:22:58.190209 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode8de49ea_a46e_4a86_81fb_734718e97b4e.slice/crio-a8f277791140f008510e983c91567fac70b3d294fbe0194939c7a95d363ce82e WatchSource:0}: Error finding container a8f277791140f008510e983c91567fac70b3d294fbe0194939c7a95d363ce82e: Status 404 returned error can't find the container with id a8f277791140f008510e983c91567fac70b3d294fbe0194939c7a95d363ce82e Jul 28 17:22:58.191569 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:58.191551 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:22:58.590411 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:22:58.590375 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" event={"ID":"e8de49ea-a46e-4a86-81fb-734718e97b4e","Type":"ContainerStarted","Data":"a8f277791140f008510e983c91567fac70b3d294fbe0194939c7a95d363ce82e"} Jul 28 17:23:01.607368 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:23:01.607328 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" event={"ID":"e8de49ea-a46e-4a86-81fb-734718e97b4e","Type":"ContainerStarted","Data":"b9472428a32bccff0b901816213a0831685b4fec1dba6a4a771a15353e099c82"} Jul 28 17:23:01.607761 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:23:01.607455 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" Jul 28 17:23:01.625752 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:23:01.625711 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" podStartSLOduration=2.307804182 podStartE2EDuration="4.625693025s" totalImagesPullingTime="2.317888843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:01.625450378 +0000 UTC m=+358.997998975" watchObservedRunningTime="2026-07-28 17:23:01.625693025 +0000 UTC m=+358.998241623" Jul 28 17:23:32.614233 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:23:32.614205 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-84fc959947-7mvnz" Jul 28 17:24:13.446713 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:13.446680 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-n52gf"] Jul 28 17:24:13.450933 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:13.450904 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-n52gf" Jul 28 17:24:13.453254 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:13.453233 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" Jul 28 17:24:13.453383 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:13.453320 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-tznh4\"" Jul 28 17:24:13.458978 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:13.458954 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-n52gf"] Jul 28 17:24:13.569596 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:13.569559 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed51d813-9bd8-4978-beb2-e65947646bbe-tls-certs\") pod \"model-serving-api-7b954ffc95-n52gf\" (UID: \"ed51d813-9bd8-4978-beb2-e65947646bbe\") " pod="kserve/model-serving-api-7b954ffc95-n52gf" Jul 28 17:24:13.569763 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:13.569619 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-498z6\" (UniqueName: \"kubernetes.io/projected/ed51d813-9bd8-4978-beb2-e65947646bbe-kube-api-access-498z6\") pod \"model-serving-api-7b954ffc95-n52gf\" (UID: \"ed51d813-9bd8-4978-beb2-e65947646bbe\") " pod="kserve/model-serving-api-7b954ffc95-n52gf" Jul 28 17:24:13.670934 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:13.670900 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498z6\" (UniqueName: \"kubernetes.io/projected/ed51d813-9bd8-4978-beb2-e65947646bbe-kube-api-access-498z6\") pod \"model-serving-api-7b954ffc95-n52gf\" (UID: \"ed51d813-9bd8-4978-beb2-e65947646bbe\") " pod="kserve/model-serving-api-7b954ffc95-n52gf" Jul 28 17:24:13.671081 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:13.670962 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed51d813-9bd8-4978-beb2-e65947646bbe-tls-certs\") pod \"model-serving-api-7b954ffc95-n52gf\" (UID: \"ed51d813-9bd8-4978-beb2-e65947646bbe\") " pod="kserve/model-serving-api-7b954ffc95-n52gf" Jul 28 17:24:13.671081 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:24:13.671059 2576 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 28 17:24:13.671150 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:24:13.671118 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed51d813-9bd8-4978-beb2-e65947646bbe-tls-certs podName:ed51d813-9bd8-4978-beb2-e65947646bbe nodeName:}" failed. No retries permitted until 2026-07-28 17:24:14.171102892 +0000 UTC m=+431.543651466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ed51d813-9bd8-4978-beb2-e65947646bbe-tls-certs") pod "model-serving-api-7b954ffc95-n52gf" (UID: "ed51d813-9bd8-4978-beb2-e65947646bbe") : secret "model-serving-api-tls" not found Jul 28 17:24:13.680311 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:13.680285 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-498z6\" (UniqueName: \"kubernetes.io/projected/ed51d813-9bd8-4978-beb2-e65947646bbe-kube-api-access-498z6\") pod \"model-serving-api-7b954ffc95-n52gf\" (UID: \"ed51d813-9bd8-4978-beb2-e65947646bbe\") " pod="kserve/model-serving-api-7b954ffc95-n52gf" Jul 28 17:24:14.174855 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:14.174823 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed51d813-9bd8-4978-beb2-e65947646bbe-tls-certs\") pod \"model-serving-api-7b954ffc95-n52gf\" (UID: \"ed51d813-9bd8-4978-beb2-e65947646bbe\") " pod="kserve/model-serving-api-7b954ffc95-n52gf" Jul 28 17:24:14.177190 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:14.177170 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed51d813-9bd8-4978-beb2-e65947646bbe-tls-certs\") pod \"model-serving-api-7b954ffc95-n52gf\" (UID: \"ed51d813-9bd8-4978-beb2-e65947646bbe\") " pod="kserve/model-serving-api-7b954ffc95-n52gf" Jul 28 17:24:14.363981 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:14.363945 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-n52gf" Jul 28 17:24:14.494696 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:14.494663 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-n52gf"] Jul 28 17:24:14.497640 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:24:14.497609 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded51d813_9bd8_4978_beb2_e65947646bbe.slice/crio-68f8df7c57f2ba57d299c98891156ad87365ead6ad0c7da95a83d6ef0b464070 WatchSource:0}: Error finding container 68f8df7c57f2ba57d299c98891156ad87365ead6ad0c7da95a83d6ef0b464070: Status 404 returned error can't find the container with id 68f8df7c57f2ba57d299c98891156ad87365ead6ad0c7da95a83d6ef0b464070 Jul 28 17:24:14.910213 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:14.910179 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-n52gf" event={"ID":"ed51d813-9bd8-4978-beb2-e65947646bbe","Type":"ContainerStarted","Data":"68f8df7c57f2ba57d299c98891156ad87365ead6ad0c7da95a83d6ef0b464070"} Jul 28 17:24:16.920672 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:16.920592 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-n52gf" event={"ID":"ed51d813-9bd8-4978-beb2-e65947646bbe","Type":"ContainerStarted","Data":"542fc17e0500065524b022204fe4a0ba23aec6a1d271337cd608e866a23584d9"} Jul 28 17:24:16.920672 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:16.920659 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-n52gf" Jul 28 17:24:16.938500 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:16.938460 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-n52gf" podStartSLOduration=1.822906738 podStartE2EDuration="3.938445276s" totalImagesPullingTime="2.115538538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:16.937656208 +0000 UTC m=+434.310204804" watchObservedRunningTime="2026-07-28 17:24:16.938445276 +0000 UTC m=+434.310993873" Jul 28 17:24:27.928755 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:24:27.928676 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-n52gf" Jul 28 17:27:03.027953 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:03.027921 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:27:03.028497 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:03.028455 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:27:31.567112 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.567075 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg"] Jul 28 17:27:31.570286 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.570269 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:27:31.577000 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.576968 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-cc402-kube-rbac-proxy-sar-config\"" Jul 28 17:27:31.577155 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.577008 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-cc402-serving-cert\"" Jul 28 17:27:31.577734 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.577714 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 17:27:31.581987 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.581964 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg"] Jul 28 17:27:31.588555 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.588534 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tdp4m\"" Jul 28 17:27:31.594317 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.594295 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0a6d7387-3385-4965-86dd-77ff40a8f2bc-proxy-tls\") pod \"switch-graph-cc402-d66f86ff-nhnxg\" (UID: \"0a6d7387-3385-4965-86dd-77ff40a8f2bc\") " pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:27:31.594439 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.594334 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a6d7387-3385-4965-86dd-77ff40a8f2bc-openshift-service-ca-bundle\") pod \"switch-graph-cc402-d66f86ff-nhnxg\" (UID: \"0a6d7387-3385-4965-86dd-77ff40a8f2bc\") " pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:27:31.694967 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.694887 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a6d7387-3385-4965-86dd-77ff40a8f2bc-openshift-service-ca-bundle\") pod \"switch-graph-cc402-d66f86ff-nhnxg\" (UID: \"0a6d7387-3385-4965-86dd-77ff40a8f2bc\") " pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:27:31.694967 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.694956 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0a6d7387-3385-4965-86dd-77ff40a8f2bc-proxy-tls\") pod \"switch-graph-cc402-d66f86ff-nhnxg\" (UID: \"0a6d7387-3385-4965-86dd-77ff40a8f2bc\") " pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:27:31.695165 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:27:31.695052 2576 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-cc402-serving-cert: secret "switch-graph-cc402-serving-cert" not found Jul 28 17:27:31.695165 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:27:31.695119 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a6d7387-3385-4965-86dd-77ff40a8f2bc-proxy-tls podName:0a6d7387-3385-4965-86dd-77ff40a8f2bc nodeName:}" failed. No retries permitted until 2026-07-28 17:27:32.195100729 +0000 UTC m=+629.567649303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/0a6d7387-3385-4965-86dd-77ff40a8f2bc-proxy-tls") pod "switch-graph-cc402-d66f86ff-nhnxg" (UID: "0a6d7387-3385-4965-86dd-77ff40a8f2bc") : secret "switch-graph-cc402-serving-cert" not found Jul 28 17:27:31.695546 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:31.695527 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a6d7387-3385-4965-86dd-77ff40a8f2bc-openshift-service-ca-bundle\") pod \"switch-graph-cc402-d66f86ff-nhnxg\" (UID: \"0a6d7387-3385-4965-86dd-77ff40a8f2bc\") " pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:27:32.200319 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:32.200279 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0a6d7387-3385-4965-86dd-77ff40a8f2bc-proxy-tls\") pod \"switch-graph-cc402-d66f86ff-nhnxg\" (UID: \"0a6d7387-3385-4965-86dd-77ff40a8f2bc\") " pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:27:32.202644 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:32.202613 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0a6d7387-3385-4965-86dd-77ff40a8f2bc-proxy-tls\") pod \"switch-graph-cc402-d66f86ff-nhnxg\" (UID: \"0a6d7387-3385-4965-86dd-77ff40a8f2bc\") " pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:27:32.487983 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:32.487892 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:27:32.623526 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:32.623492 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg"] Jul 28 17:27:32.626489 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:27:32.626455 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6d7387_3385_4965_86dd_77ff40a8f2bc.slice/crio-78bd7de13b1fd18a10d2579ffa477b587cc88ef4850328fad2546c9cb54d9157 WatchSource:0}: Error finding container 78bd7de13b1fd18a10d2579ffa477b587cc88ef4850328fad2546c9cb54d9157: Status 404 returned error can't find the container with id 78bd7de13b1fd18a10d2579ffa477b587cc88ef4850328fad2546c9cb54d9157 Jul 28 17:27:32.755523 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:32.755430 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" event={"ID":"0a6d7387-3385-4965-86dd-77ff40a8f2bc","Type":"ContainerStarted","Data":"78bd7de13b1fd18a10d2579ffa477b587cc88ef4850328fad2546c9cb54d9157"} Jul 28 17:27:35.771886 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:35.771850 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" event={"ID":"0a6d7387-3385-4965-86dd-77ff40a8f2bc","Type":"ContainerStarted","Data":"5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0"} Jul 28 17:27:35.772306 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:35.771947 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:27:35.789866 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:35.789816 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" podStartSLOduration=2.126817486 podStartE2EDuration="4.789798742s" totalImagesPullingTime="2.662981256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:27:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:27:35.788751836 +0000 UTC m=+633.161300432" watchObservedRunningTime="2026-07-28 17:27:35.789798742 +0000 UTC m=+633.162347329" Jul 28 17:27:41.789740 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:41.785345 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:27:45.720462 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:45.720422 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg"] Jul 28 17:27:45.720931 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:45.720633 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" podUID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" containerName="switch-graph-cc402" containerID="cri-o://5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0" gracePeriod=30 Jul 28 17:27:46.783012 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:46.782967 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" podUID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" containerName="switch-graph-cc402" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:27:51.782978 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:51.782940 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" podUID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" containerName="switch-graph-cc402" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:27:56.783487 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:56.783448 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" podUID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" containerName="switch-graph-cc402" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:27:56.783921 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:27:56.783566 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:28:01.783149 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:01.783108 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" podUID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" containerName="switch-graph-cc402" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:28:06.782952 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:06.782914 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" podUID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" containerName="switch-graph-cc402" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:28:11.783374 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:11.783335 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" podUID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" containerName="switch-graph-cc402" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:28:15.924892 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:15.924869 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:28:15.947040 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:15.947011 2576 generic.go:356] "Generic (PLEG): container finished" podID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" containerID="5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0" exitCode=137 Jul 28 17:28:15.947160 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:15.947054 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" event={"ID":"0a6d7387-3385-4965-86dd-77ff40a8f2bc","Type":"ContainerDied","Data":"5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0"} Jul 28 17:28:15.947160 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:15.947074 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" Jul 28 17:28:15.947160 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:15.947126 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg" event={"ID":"0a6d7387-3385-4965-86dd-77ff40a8f2bc","Type":"ContainerDied","Data":"78bd7de13b1fd18a10d2579ffa477b587cc88ef4850328fad2546c9cb54d9157"} Jul 28 17:28:15.947160 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:15.947149 2576 scope.go:117] "RemoveContainer" containerID="5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0" Jul 28 17:28:15.957355 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:15.957324 2576 scope.go:117] "RemoveContainer" containerID="5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0" Jul 28 17:28:15.957607 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:28:15.957589 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0\": container with ID starting with 5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0 not found: ID does not exist" containerID="5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0" Jul 28 17:28:15.957676 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:15.957619 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0"} err="failed to get container status \"5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0\": rpc error: code = NotFound desc = could not find container \"5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0\": container with ID starting with 5ec75094592603327a7f2cf153a8bcb3aa2e307fea7f857a73ee260e39fce2e0 not found: ID does not exist" Jul 28 17:28:16.032040 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:16.032006 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0a6d7387-3385-4965-86dd-77ff40a8f2bc-proxy-tls\") pod \"0a6d7387-3385-4965-86dd-77ff40a8f2bc\" (UID: \"0a6d7387-3385-4965-86dd-77ff40a8f2bc\") " Jul 28 17:28:16.032198 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:16.032074 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a6d7387-3385-4965-86dd-77ff40a8f2bc-openshift-service-ca-bundle\") pod \"0a6d7387-3385-4965-86dd-77ff40a8f2bc\" (UID: \"0a6d7387-3385-4965-86dd-77ff40a8f2bc\") " Jul 28 17:28:16.032417 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:16.032381 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a6d7387-3385-4965-86dd-77ff40a8f2bc-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "0a6d7387-3385-4965-86dd-77ff40a8f2bc" (UID: "0a6d7387-3385-4965-86dd-77ff40a8f2bc"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:28:16.034228 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:16.034202 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a6d7387-3385-4965-86dd-77ff40a8f2bc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0a6d7387-3385-4965-86dd-77ff40a8f2bc" (UID: "0a6d7387-3385-4965-86dd-77ff40a8f2bc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:28:16.133500 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:16.133456 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a6d7387-3385-4965-86dd-77ff40a8f2bc-openshift-service-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:28:16.133500 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:16.133497 2576 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0a6d7387-3385-4965-86dd-77ff40a8f2bc-proxy-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:28:16.269691 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:16.269665 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg"] Jul 28 17:28:16.273806 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:16.273769 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-cc402-d66f86ff-nhnxg"] Jul 28 17:28:17.143604 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:17.143568 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" path="/var/lib/kubelet/pods/0a6d7387-3385-4965-86dd-77ff40a8f2bc/volumes" Jul 28 17:28:31.544897 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.544863 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd"] Jul 28 17:28:31.545260 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.545184 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" containerName="switch-graph-cc402" Jul 28 17:28:31.545260 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.545195 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" containerName="switch-graph-cc402" Jul 28 17:28:31.545260 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.545244 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a6d7387-3385-4965-86dd-77ff40a8f2bc" containerName="switch-graph-cc402" Jul 28 17:28:31.547949 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.547929 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:28:31.550402 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.550376 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Jul 28 17:28:31.550628 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.550607 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Jul 28 17:28:31.550688 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.550653 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 17:28:31.550741 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.550692 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tdp4m\"" Jul 28 17:28:31.556605 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.556581 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd"] Jul 28 17:28:31.646643 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.646612 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d554db6a-5bc5-474f-b208-cf63d59d55bc-openshift-service-ca-bundle\") pod \"model-chainer-c5b4df566-hwmdd\" (UID: \"d554db6a-5bc5-474f-b208-cf63d59d55bc\") " pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:28:31.646759 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.646651 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d554db6a-5bc5-474f-b208-cf63d59d55bc-proxy-tls\") pod \"model-chainer-c5b4df566-hwmdd\" (UID: \"d554db6a-5bc5-474f-b208-cf63d59d55bc\") " pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:28:31.747028 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.747003 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d554db6a-5bc5-474f-b208-cf63d59d55bc-proxy-tls\") pod \"model-chainer-c5b4df566-hwmdd\" (UID: \"d554db6a-5bc5-474f-b208-cf63d59d55bc\") " pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:28:31.747137 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.747080 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d554db6a-5bc5-474f-b208-cf63d59d55bc-openshift-service-ca-bundle\") pod \"model-chainer-c5b4df566-hwmdd\" (UID: \"d554db6a-5bc5-474f-b208-cf63d59d55bc\") " pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:28:31.747682 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.747660 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d554db6a-5bc5-474f-b208-cf63d59d55bc-openshift-service-ca-bundle\") pod \"model-chainer-c5b4df566-hwmdd\" (UID: \"d554db6a-5bc5-474f-b208-cf63d59d55bc\") " pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:28:31.749300 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.749262 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d554db6a-5bc5-474f-b208-cf63d59d55bc-proxy-tls\") pod \"model-chainer-c5b4df566-hwmdd\" (UID: \"d554db6a-5bc5-474f-b208-cf63d59d55bc\") " pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:28:31.858966 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.858911 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:28:31.988420 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:31.988384 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd"] Jul 28 17:28:31.990940 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:28:31.990910 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd554db6a_5bc5_474f_b208_cf63d59d55bc.slice/crio-e1ab242ae18db90f43f191410c52133ccfbacd079076e5ecb6c3effa5841ac01 WatchSource:0}: Error finding container e1ab242ae18db90f43f191410c52133ccfbacd079076e5ecb6c3effa5841ac01: Status 404 returned error can't find the container with id e1ab242ae18db90f43f191410c52133ccfbacd079076e5ecb6c3effa5841ac01 Jul 28 17:28:32.015287 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:32.015258 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" event={"ID":"d554db6a-5bc5-474f-b208-cf63d59d55bc","Type":"ContainerStarted","Data":"e1ab242ae18db90f43f191410c52133ccfbacd079076e5ecb6c3effa5841ac01"} Jul 28 17:28:33.020902 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:33.020860 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" event={"ID":"d554db6a-5bc5-474f-b208-cf63d59d55bc","Type":"ContainerStarted","Data":"9ececc20c54f49ce3c72d3fcd9dd255e9c02425b94ef3d020b20e6509a657fab"} Jul 28 17:28:33.021295 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:33.020974 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:28:33.040761 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:33.040721 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" podStartSLOduration=2.040707121 podStartE2EDuration="2.040707121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:28:33.039045183 +0000 UTC m=+690.411593779" watchObservedRunningTime="2026-07-28 17:28:33.040707121 +0000 UTC m=+690.413255716" Jul 28 17:28:39.031913 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:39.031883 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:28:41.659681 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:41.659648 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd"] Jul 28 17:28:41.660129 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:41.659917 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" podUID="d554db6a-5bc5-474f-b208-cf63d59d55bc" containerName="model-chainer" containerID="cri-o://9ececc20c54f49ce3c72d3fcd9dd255e9c02425b94ef3d020b20e6509a657fab" gracePeriod=30 Jul 28 17:28:44.030187 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:44.030146 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" podUID="d554db6a-5bc5-474f-b208-cf63d59d55bc" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:28:45.969455 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:45.969422 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq"] Jul 28 17:28:45.972765 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:45.972742 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:28:45.975267 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:45.975247 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-65026-serving-cert\"" Jul 28 17:28:45.975373 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:45.975251 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-65026-kube-rbac-proxy-sar-config\"" Jul 28 17:28:45.982921 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:45.982899 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq"] Jul 28 17:28:46.051983 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:46.051952 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-openshift-service-ca-bundle\") pod \"switch-graph-65026-6f5d6b7d79-snrnq\" (UID: \"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5\") " pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:28:46.052115 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:46.051992 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-proxy-tls\") pod \"switch-graph-65026-6f5d6b7d79-snrnq\" (UID: \"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5\") " pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:28:46.152598 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:46.152563 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-openshift-service-ca-bundle\") pod \"switch-graph-65026-6f5d6b7d79-snrnq\" (UID: \"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5\") " pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:28:46.152750 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:46.152644 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-proxy-tls\") pod \"switch-graph-65026-6f5d6b7d79-snrnq\" (UID: \"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5\") " pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:28:46.152750 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:28:46.152726 2576 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-65026-serving-cert: secret "switch-graph-65026-serving-cert" not found Jul 28 17:28:46.152941 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:28:46.152921 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-proxy-tls podName:49e6f64e-1f2a-4932-bc45-c98ddc93a5e5 nodeName:}" failed. No retries permitted until 2026-07-28 17:28:46.652890653 +0000 UTC m=+704.025439227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-proxy-tls") pod "switch-graph-65026-6f5d6b7d79-snrnq" (UID: "49e6f64e-1f2a-4932-bc45-c98ddc93a5e5") : secret "switch-graph-65026-serving-cert" not found Jul 28 17:28:46.153300 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:46.153281 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-openshift-service-ca-bundle\") pod \"switch-graph-65026-6f5d6b7d79-snrnq\" (UID: \"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5\") " pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:28:46.657353 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:46.657311 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-proxy-tls\") pod \"switch-graph-65026-6f5d6b7d79-snrnq\" (UID: \"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5\") " pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:28:46.659624 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:46.659599 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-proxy-tls\") pod \"switch-graph-65026-6f5d6b7d79-snrnq\" (UID: \"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5\") " pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:28:46.883312 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:46.883284 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:28:47.012593 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:47.012570 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq"] Jul 28 17:28:47.015396 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:28:47.015366 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e6f64e_1f2a_4932_bc45_c98ddc93a5e5.slice/crio-a1d4a22fe390f0d0f70b7a8e4d9d8a45b39592bfe8e989f4e88c3ad98aa2d5e4 WatchSource:0}: Error finding container a1d4a22fe390f0d0f70b7a8e4d9d8a45b39592bfe8e989f4e88c3ad98aa2d5e4: Status 404 returned error can't find the container with id a1d4a22fe390f0d0f70b7a8e4d9d8a45b39592bfe8e989f4e88c3ad98aa2d5e4 Jul 28 17:28:47.081233 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:47.081203 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" event={"ID":"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5","Type":"ContainerStarted","Data":"a1d4a22fe390f0d0f70b7a8e4d9d8a45b39592bfe8e989f4e88c3ad98aa2d5e4"} Jul 28 17:28:48.086429 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:48.086391 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" event={"ID":"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5","Type":"ContainerStarted","Data":"4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c"} Jul 28 17:28:48.086774 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:48.086473 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:28:48.104226 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:48.104175 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" podStartSLOduration=3.104161767 podStartE2EDuration="3.104161767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:28:48.103204348 +0000 UTC m=+705.475752945" watchObservedRunningTime="2026-07-28 17:28:48.104161767 +0000 UTC m=+705.476710362" Jul 28 17:28:49.029289 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:49.029251 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" podUID="d554db6a-5bc5-474f-b208-cf63d59d55bc" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:28:54.029710 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:54.029661 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" podUID="d554db6a-5bc5-474f-b208-cf63d59d55bc" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:28:54.030159 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:54.029773 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:28:54.096292 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:54.096267 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:28:59.030236 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:28:59.030154 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" podUID="d554db6a-5bc5-474f-b208-cf63d59d55bc" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:29:04.029401 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:04.029363 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" podUID="d554db6a-5bc5-474f-b208-cf63d59d55bc" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:29:09.029189 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:09.029147 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" podUID="d554db6a-5bc5-474f-b208-cf63d59d55bc" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:29:12.191603 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:12.191576 2576 generic.go:356] "Generic (PLEG): container finished" podID="d554db6a-5bc5-474f-b208-cf63d59d55bc" containerID="9ececc20c54f49ce3c72d3fcd9dd255e9c02425b94ef3d020b20e6509a657fab" exitCode=0 Jul 28 17:29:12.191989 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:12.191647 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" event={"ID":"d554db6a-5bc5-474f-b208-cf63d59d55bc","Type":"ContainerDied","Data":"9ececc20c54f49ce3c72d3fcd9dd255e9c02425b94ef3d020b20e6509a657fab"} Jul 28 17:29:12.297628 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:12.297607 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:29:12.362315 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:12.362296 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d554db6a-5bc5-474f-b208-cf63d59d55bc-proxy-tls\") pod \"d554db6a-5bc5-474f-b208-cf63d59d55bc\" (UID: \"d554db6a-5bc5-474f-b208-cf63d59d55bc\") " Jul 28 17:29:12.362426 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:12.362384 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d554db6a-5bc5-474f-b208-cf63d59d55bc-openshift-service-ca-bundle\") pod \"d554db6a-5bc5-474f-b208-cf63d59d55bc\" (UID: \"d554db6a-5bc5-474f-b208-cf63d59d55bc\") " Jul 28 17:29:12.362657 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:12.362638 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d554db6a-5bc5-474f-b208-cf63d59d55bc-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "d554db6a-5bc5-474f-b208-cf63d59d55bc" (UID: "d554db6a-5bc5-474f-b208-cf63d59d55bc"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:29:12.364128 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:12.364109 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d554db6a-5bc5-474f-b208-cf63d59d55bc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d554db6a-5bc5-474f-b208-cf63d59d55bc" (UID: "d554db6a-5bc5-474f-b208-cf63d59d55bc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:29:12.463770 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:12.463719 2576 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d554db6a-5bc5-474f-b208-cf63d59d55bc-proxy-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:29:12.463770 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:12.463740 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d554db6a-5bc5-474f-b208-cf63d59d55bc-openshift-service-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:29:13.196928 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:13.196901 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" Jul 28 17:29:13.196928 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:13.196910 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd" event={"ID":"d554db6a-5bc5-474f-b208-cf63d59d55bc","Type":"ContainerDied","Data":"e1ab242ae18db90f43f191410c52133ccfbacd079076e5ecb6c3effa5841ac01"} Jul 28 17:29:13.197342 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:13.196959 2576 scope.go:117] "RemoveContainer" containerID="9ececc20c54f49ce3c72d3fcd9dd255e9c02425b94ef3d020b20e6509a657fab" Jul 28 17:29:13.215373 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:13.215351 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd"] Jul 28 17:29:13.219432 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:13.219414 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-c5b4df566-hwmdd"] Jul 28 17:29:15.143415 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:15.143383 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d554db6a-5bc5-474f-b208-cf63d59d55bc" path="/var/lib/kubelet/pods/d554db6a-5bc5-474f-b208-cf63d59d55bc/volumes" Jul 28 17:29:41.886412 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:41.886375 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8"] Jul 28 17:29:41.886858 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:41.886842 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d554db6a-5bc5-474f-b208-cf63d59d55bc" containerName="model-chainer" Jul 28 17:29:41.886907 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:41.886860 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d554db6a-5bc5-474f-b208-cf63d59d55bc" containerName="model-chainer" Jul 28 17:29:41.886944 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:41.886936 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d554db6a-5bc5-474f-b208-cf63d59d55bc" containerName="model-chainer" Jul 28 17:29:41.890424 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:41.890395 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:29:41.896321 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:41.896294 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-8c20d-serving-cert\"" Jul 28 17:29:41.896469 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:41.896293 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-8c20d-kube-rbac-proxy-sar-config\"" Jul 28 17:29:41.899386 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:41.899366 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8"] Jul 28 17:29:41.989416 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:41.989395 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4be8c2c-41c8-48d9-9feb-0848236b55e5-proxy-tls\") pod \"sequence-graph-8c20d-75b48d7b6d-dzgs8\" (UID: \"b4be8c2c-41c8-48d9-9feb-0848236b55e5\") " pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:29:41.989514 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:41.989425 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4be8c2c-41c8-48d9-9feb-0848236b55e5-openshift-service-ca-bundle\") pod \"sequence-graph-8c20d-75b48d7b6d-dzgs8\" (UID: \"b4be8c2c-41c8-48d9-9feb-0848236b55e5\") " pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:29:42.090649 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:42.090624 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4be8c2c-41c8-48d9-9feb-0848236b55e5-proxy-tls\") pod \"sequence-graph-8c20d-75b48d7b6d-dzgs8\" (UID: \"b4be8c2c-41c8-48d9-9feb-0848236b55e5\") " pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:29:42.090812 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:42.090657 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4be8c2c-41c8-48d9-9feb-0848236b55e5-openshift-service-ca-bundle\") pod \"sequence-graph-8c20d-75b48d7b6d-dzgs8\" (UID: \"b4be8c2c-41c8-48d9-9feb-0848236b55e5\") " pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:29:42.090812 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:29:42.090768 2576 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-8c20d-serving-cert: secret "sequence-graph-8c20d-serving-cert" not found Jul 28 17:29:42.090924 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:29:42.090841 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4be8c2c-41c8-48d9-9feb-0848236b55e5-proxy-tls podName:b4be8c2c-41c8-48d9-9feb-0848236b55e5 nodeName:}" failed. No retries permitted until 2026-07-28 17:29:42.590826799 +0000 UTC m=+759.963375372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b4be8c2c-41c8-48d9-9feb-0848236b55e5-proxy-tls") pod "sequence-graph-8c20d-75b48d7b6d-dzgs8" (UID: "b4be8c2c-41c8-48d9-9feb-0848236b55e5") : secret "sequence-graph-8c20d-serving-cert" not found Jul 28 17:29:42.091200 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:42.091183 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4be8c2c-41c8-48d9-9feb-0848236b55e5-openshift-service-ca-bundle\") pod \"sequence-graph-8c20d-75b48d7b6d-dzgs8\" (UID: \"b4be8c2c-41c8-48d9-9feb-0848236b55e5\") " pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:29:42.593497 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:42.593465 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4be8c2c-41c8-48d9-9feb-0848236b55e5-proxy-tls\") pod \"sequence-graph-8c20d-75b48d7b6d-dzgs8\" (UID: \"b4be8c2c-41c8-48d9-9feb-0848236b55e5\") " pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:29:42.595743 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:42.595723 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4be8c2c-41c8-48d9-9feb-0848236b55e5-proxy-tls\") pod \"sequence-graph-8c20d-75b48d7b6d-dzgs8\" (UID: \"b4be8c2c-41c8-48d9-9feb-0848236b55e5\") " pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:29:42.806518 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:42.806490 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:29:42.935576 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:42.935553 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8"] Jul 28 17:29:42.938148 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:29:42.938121 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4be8c2c_41c8_48d9_9feb_0848236b55e5.slice/crio-d6118392dd4b3164ede76a2e3970d764773afab02bff377b37e9cb383d15c2f8 WatchSource:0}: Error finding container d6118392dd4b3164ede76a2e3970d764773afab02bff377b37e9cb383d15c2f8: Status 404 returned error can't find the container with id d6118392dd4b3164ede76a2e3970d764773afab02bff377b37e9cb383d15c2f8 Jul 28 17:29:43.319324 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:43.319291 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" event={"ID":"b4be8c2c-41c8-48d9-9feb-0848236b55e5","Type":"ContainerStarted","Data":"baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11"} Jul 28 17:29:43.319495 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:43.319371 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" event={"ID":"b4be8c2c-41c8-48d9-9feb-0848236b55e5","Type":"ContainerStarted","Data":"d6118392dd4b3164ede76a2e3970d764773afab02bff377b37e9cb383d15c2f8"} Jul 28 17:29:43.319495 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:43.319391 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:29:43.339252 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:43.339062 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" podStartSLOduration=2.339046726 podStartE2EDuration="2.339046726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:29:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:29:43.338989361 +0000 UTC m=+760.711537957" watchObservedRunningTime="2026-07-28 17:29:43.339046726 +0000 UTC m=+760.711595321" Jul 28 17:29:49.329189 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:29:49.329158 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:32:03.058938 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:32:03.058858 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:32:03.060197 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:32:03.060171 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:37:00.547897 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:00.547816 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq"] Jul 28 17:37:00.548403 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:00.548063 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" podUID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" containerName="switch-graph-65026" containerID="cri-o://4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c" gracePeriod=30 Jul 28 17:37:03.088865 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:03.088835 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:37:03.090318 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:03.090296 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:37:04.095024 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:04.094987 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" podUID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" containerName="switch-graph-65026" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:37:09.095224 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:09.095186 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" podUID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" containerName="switch-graph-65026" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:37:14.095085 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:14.095045 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" podUID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" containerName="switch-graph-65026" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:37:14.095461 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:14.095155 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:37:19.095154 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:19.095109 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" podUID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" containerName="switch-graph-65026" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:37:24.095056 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:24.095020 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" podUID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" containerName="switch-graph-65026" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:37:29.094303 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:29.094263 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" podUID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" containerName="switch-graph-65026" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:37:30.690706 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:30.690667 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:37:30.801397 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:30.801366 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-proxy-tls\") pod \"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5\" (UID: \"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5\") " Jul 28 17:37:30.801546 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:30.801404 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-openshift-service-ca-bundle\") pod \"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5\" (UID: \"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5\") " Jul 28 17:37:30.801852 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:30.801766 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" (UID: "49e6f64e-1f2a-4932-bc45-c98ddc93a5e5"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:37:30.803289 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:30.803262 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" (UID: "49e6f64e-1f2a-4932-bc45-c98ddc93a5e5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:37:30.902348 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:30.902321 2576 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-proxy-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:37:30.902348 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:30.902344 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5-openshift-service-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:37:31.302017 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:31.301981 2576 generic.go:356] "Generic (PLEG): container finished" podID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" containerID="4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c" exitCode=0 Jul 28 17:37:31.302242 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:31.302025 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" Jul 28 17:37:31.302242 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:31.302043 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" event={"ID":"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5","Type":"ContainerDied","Data":"4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c"} Jul 28 17:37:31.302242 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:31.302084 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq" event={"ID":"49e6f64e-1f2a-4932-bc45-c98ddc93a5e5","Type":"ContainerDied","Data":"a1d4a22fe390f0d0f70b7a8e4d9d8a45b39592bfe8e989f4e88c3ad98aa2d5e4"} Jul 28 17:37:31.302242 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:31.302106 2576 scope.go:117] "RemoveContainer" containerID="4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c" Jul 28 17:37:31.309955 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:31.309935 2576 scope.go:117] "RemoveContainer" containerID="4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c" Jul 28 17:37:31.310200 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:37:31.310184 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c\": container with ID starting with 4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c not found: ID does not exist" containerID="4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c" Jul 28 17:37:31.310257 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:31.310207 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c"} err="failed to get container status \"4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c\": rpc error: code = NotFound desc = could not find container \"4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c\": container with ID starting with 4ad8b9c84ea00b76ce952f2dd9693b0ca019c39a1d08682b13b701f7f0693f2c not found: ID does not exist" Jul 28 17:37:31.319118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:31.319095 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq"] Jul 28 17:37:31.322890 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:31.322870 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-65026-6f5d6b7d79-snrnq"] Jul 28 17:37:33.143379 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:33.143346 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" path="/var/lib/kubelet/pods/49e6f64e-1f2a-4932-bc45-c98ddc93a5e5/volumes" Jul 28 17:37:56.578499 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:56.578425 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8"] Jul 28 17:37:56.579047 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:56.578606 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" podUID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" containerName="sequence-graph-8c20d" containerID="cri-o://baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11" gracePeriod=30 Jul 28 17:37:59.328482 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:37:59.328436 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" podUID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" containerName="sequence-graph-8c20d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:00.792616 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.792578 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh"] Jul 28 17:38:00.793121 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.793101 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" containerName="switch-graph-65026" Jul 28 17:38:00.793188 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.793125 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" containerName="switch-graph-65026" Jul 28 17:38:00.793243 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.793209 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e6f64e-1f2a-4932-bc45-c98ddc93a5e5" containerName="switch-graph-65026" Jul 28 17:38:00.797682 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.797660 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:00.800175 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.800153 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-59e5e-kube-rbac-proxy-sar-config\"" Jul 28 17:38:00.800307 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.800202 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-59e5e-serving-cert\"" Jul 28 17:38:00.803093 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.803069 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh"] Jul 28 17:38:00.831023 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.830988 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-proxy-tls\") pod \"ensemble-graph-59e5e-6fc8d76f64-fw7jh\" (UID: \"f1753bc9-60d2-4e29-8a88-175f5d08c3c9\") " pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:00.831199 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.831030 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-openshift-service-ca-bundle\") pod \"ensemble-graph-59e5e-6fc8d76f64-fw7jh\" (UID: \"f1753bc9-60d2-4e29-8a88-175f5d08c3c9\") " pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:00.931684 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.931653 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-proxy-tls\") pod \"ensemble-graph-59e5e-6fc8d76f64-fw7jh\" (UID: \"f1753bc9-60d2-4e29-8a88-175f5d08c3c9\") " pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:00.931684 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.931697 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-openshift-service-ca-bundle\") pod \"ensemble-graph-59e5e-6fc8d76f64-fw7jh\" (UID: \"f1753bc9-60d2-4e29-8a88-175f5d08c3c9\") " pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:00.932397 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.932377 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-openshift-service-ca-bundle\") pod \"ensemble-graph-59e5e-6fc8d76f64-fw7jh\" (UID: \"f1753bc9-60d2-4e29-8a88-175f5d08c3c9\") " pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:00.934085 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:00.934066 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-proxy-tls\") pod \"ensemble-graph-59e5e-6fc8d76f64-fw7jh\" (UID: \"f1753bc9-60d2-4e29-8a88-175f5d08c3c9\") " pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:01.108063 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:01.107976 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:01.236215 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:01.236183 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh"] Jul 28 17:38:01.239476 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:38:01.239450 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1753bc9_60d2_4e29_8a88_175f5d08c3c9.slice/crio-9624011835bb6fe3202d0e16e661206eabc842a3c3d34cbf578911df84e46b7c WatchSource:0}: Error finding container 9624011835bb6fe3202d0e16e661206eabc842a3c3d34cbf578911df84e46b7c: Status 404 returned error can't find the container with id 9624011835bb6fe3202d0e16e661206eabc842a3c3d34cbf578911df84e46b7c Jul 28 17:38:01.429416 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:01.429324 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" event={"ID":"f1753bc9-60d2-4e29-8a88-175f5d08c3c9","Type":"ContainerStarted","Data":"01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff"} Jul 28 17:38:01.429416 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:01.429375 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" event={"ID":"f1753bc9-60d2-4e29-8a88-175f5d08c3c9","Type":"ContainerStarted","Data":"9624011835bb6fe3202d0e16e661206eabc842a3c3d34cbf578911df84e46b7c"} Jul 28 17:38:01.429416 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:01.429414 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:01.445712 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:01.445663 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" podStartSLOduration=1.445647243 podStartE2EDuration="1.445647243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:38:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:38:01.444773659 +0000 UTC m=+1258.817322266" watchObservedRunningTime="2026-07-28 17:38:01.445647243 +0000 UTC m=+1258.818195851" Jul 28 17:38:04.327846 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:04.327778 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" podUID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" containerName="sequence-graph-8c20d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:07.439649 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:07.439618 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:09.327906 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:09.327866 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" podUID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" containerName="sequence-graph-8c20d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:09.328396 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:09.327984 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:38:10.868056 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:10.867977 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh"] Jul 28 17:38:10.868483 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:10.868241 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" podUID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" containerName="ensemble-graph-59e5e" containerID="cri-o://01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff" gracePeriod=30 Jul 28 17:38:12.438477 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:12.438434 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" podUID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" containerName="ensemble-graph-59e5e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:14.328164 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:14.328127 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" podUID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" containerName="sequence-graph-8c20d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:17.438607 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:17.438562 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" podUID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" containerName="ensemble-graph-59e5e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:19.328456 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:19.328415 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" podUID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" containerName="sequence-graph-8c20d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:22.438360 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:22.438318 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" podUID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" containerName="ensemble-graph-59e5e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:22.438843 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:22.438414 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:24.328024 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:24.327986 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" podUID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" containerName="sequence-graph-8c20d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:27.225932 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.225908 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:38:27.348071 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.348005 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4be8c2c-41c8-48d9-9feb-0848236b55e5-openshift-service-ca-bundle\") pod \"b4be8c2c-41c8-48d9-9feb-0848236b55e5\" (UID: \"b4be8c2c-41c8-48d9-9feb-0848236b55e5\") " Jul 28 17:38:27.348191 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.348107 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4be8c2c-41c8-48d9-9feb-0848236b55e5-proxy-tls\") pod \"b4be8c2c-41c8-48d9-9feb-0848236b55e5\" (UID: \"b4be8c2c-41c8-48d9-9feb-0848236b55e5\") " Jul 28 17:38:27.348344 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.348322 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4be8c2c-41c8-48d9-9feb-0848236b55e5-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b4be8c2c-41c8-48d9-9feb-0848236b55e5" (UID: "b4be8c2c-41c8-48d9-9feb-0848236b55e5"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:38:27.350058 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.350036 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4be8c2c-41c8-48d9-9feb-0848236b55e5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b4be8c2c-41c8-48d9-9feb-0848236b55e5" (UID: "b4be8c2c-41c8-48d9-9feb-0848236b55e5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:38:27.438097 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.438069 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" podUID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" containerName="ensemble-graph-59e5e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:27.449005 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.448984 2576 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4be8c2c-41c8-48d9-9feb-0848236b55e5-proxy-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:38:27.449068 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.449005 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4be8c2c-41c8-48d9-9feb-0848236b55e5-openshift-service-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:38:27.543814 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.543763 2576 generic.go:356] "Generic (PLEG): container finished" podID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" containerID="baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11" exitCode=0 Jul 28 17:38:27.544171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.543820 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" Jul 28 17:38:27.544171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.543855 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" event={"ID":"b4be8c2c-41c8-48d9-9feb-0848236b55e5","Type":"ContainerDied","Data":"baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11"} Jul 28 17:38:27.544171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.543914 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8" event={"ID":"b4be8c2c-41c8-48d9-9feb-0848236b55e5","Type":"ContainerDied","Data":"d6118392dd4b3164ede76a2e3970d764773afab02bff377b37e9cb383d15c2f8"} Jul 28 17:38:27.544171 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.543946 2576 scope.go:117] "RemoveContainer" containerID="baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11" Jul 28 17:38:27.552421 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.552401 2576 scope.go:117] "RemoveContainer" containerID="baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11" Jul 28 17:38:27.552685 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:38:27.552668 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11\": container with ID starting with baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11 not found: ID does not exist" containerID="baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11" Jul 28 17:38:27.552726 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.552694 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11"} err="failed to get container status \"baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11\": rpc error: code = NotFound desc = could not find container \"baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11\": container with ID starting with baff0ba06eb1ec65ee4e6bf7a86b8483eaae129a2b91cc3cbed2bd5c2f422c11 not found: ID does not exist" Jul 28 17:38:27.565749 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.565664 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8"] Jul 28 17:38:27.573925 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:27.570547 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8c20d-75b48d7b6d-dzgs8"] Jul 28 17:38:29.143519 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:29.143484 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" path="/var/lib/kubelet/pods/b4be8c2c-41c8-48d9-9feb-0848236b55e5/volumes" Jul 28 17:38:32.438450 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:32.438410 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" podUID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" containerName="ensemble-graph-59e5e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:37.438173 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:37.438136 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" podUID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" containerName="ensemble-graph-59e5e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:38:41.015422 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.015394 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:41.153693 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.153619 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-proxy-tls\") pod \"f1753bc9-60d2-4e29-8a88-175f5d08c3c9\" (UID: \"f1753bc9-60d2-4e29-8a88-175f5d08c3c9\") " Jul 28 17:38:41.153833 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.153776 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-openshift-service-ca-bundle\") pod \"f1753bc9-60d2-4e29-8a88-175f5d08c3c9\" (UID: \"f1753bc9-60d2-4e29-8a88-175f5d08c3c9\") " Jul 28 17:38:41.154079 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.154052 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "f1753bc9-60d2-4e29-8a88-175f5d08c3c9" (UID: "f1753bc9-60d2-4e29-8a88-175f5d08c3c9"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:38:41.155568 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.155547 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f1753bc9-60d2-4e29-8a88-175f5d08c3c9" (UID: "f1753bc9-60d2-4e29-8a88-175f5d08c3c9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:38:41.254366 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.254338 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-openshift-service-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:38:41.254478 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.254369 2576 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1753bc9-60d2-4e29-8a88-175f5d08c3c9-proxy-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:38:41.606177 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.606139 2576 generic.go:356] "Generic (PLEG): container finished" podID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" containerID="01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff" exitCode=0 Jul 28 17:38:41.606315 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.606182 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" Jul 28 17:38:41.606315 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.606195 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" event={"ID":"f1753bc9-60d2-4e29-8a88-175f5d08c3c9","Type":"ContainerDied","Data":"01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff"} Jul 28 17:38:41.606315 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.606250 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh" event={"ID":"f1753bc9-60d2-4e29-8a88-175f5d08c3c9","Type":"ContainerDied","Data":"9624011835bb6fe3202d0e16e661206eabc842a3c3d34cbf578911df84e46b7c"} Jul 28 17:38:41.606315 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.606275 2576 scope.go:117] "RemoveContainer" containerID="01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff" Jul 28 17:38:41.615237 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.615221 2576 scope.go:117] "RemoveContainer" containerID="01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff" Jul 28 17:38:41.615477 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:38:41.615456 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff\": container with ID starting with 01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff not found: ID does not exist" containerID="01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff" Jul 28 17:38:41.615527 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.615485 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff"} err="failed to get container status \"01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff\": rpc error: code = NotFound desc = could not find container \"01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff\": container with ID starting with 01ab67ceb59c3c6249471a1bc2d25a9098ecd09e709756ec73c135ee574988ff not found: ID does not exist" Jul 28 17:38:41.627383 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.627363 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh"] Jul 28 17:38:41.631318 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:41.631300 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-59e5e-6fc8d76f64-fw7jh"] Jul 28 17:38:43.144025 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:43.143987 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" path="/var/lib/kubelet/pods/f1753bc9-60d2-4e29-8a88-175f5d08c3c9/volumes" Jul 28 17:38:56.784709 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.784666 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn"] Jul 28 17:38:56.785175 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.785136 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" containerName="ensemble-graph-59e5e" Jul 28 17:38:56.785175 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.785155 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" containerName="ensemble-graph-59e5e" Jul 28 17:38:56.785175 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.785169 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" containerName="sequence-graph-8c20d" Jul 28 17:38:56.785335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.785178 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" containerName="sequence-graph-8c20d" Jul 28 17:38:56.785335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.785264 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4be8c2c-41c8-48d9-9feb-0848236b55e5" containerName="sequence-graph-8c20d" Jul 28 17:38:56.785335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.785279 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1753bc9-60d2-4e29-8a88-175f5d08c3c9" containerName="ensemble-graph-59e5e" Jul 28 17:38:56.789470 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.789448 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:38:56.791930 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.791894 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-4108d-kube-rbac-proxy-sar-config\"" Jul 28 17:38:56.792054 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.791938 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tdp4m\"" Jul 28 17:38:56.792054 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.791969 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-4108d-serving-cert\"" Jul 28 17:38:56.792054 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.791995 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 17:38:56.795952 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.795929 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn"] Jul 28 17:38:56.962803 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.962750 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65a3f603-5eb5-442c-a410-893923546566-openshift-service-ca-bundle\") pod \"sequence-graph-4108d-594769977b-sx7fn\" (UID: \"65a3f603-5eb5-442c-a410-893923546566\") " pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:38:56.962966 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:56.962835 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/65a3f603-5eb5-442c-a410-893923546566-proxy-tls\") pod \"sequence-graph-4108d-594769977b-sx7fn\" (UID: \"65a3f603-5eb5-442c-a410-893923546566\") " pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:38:57.064101 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:57.064023 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65a3f603-5eb5-442c-a410-893923546566-openshift-service-ca-bundle\") pod \"sequence-graph-4108d-594769977b-sx7fn\" (UID: \"65a3f603-5eb5-442c-a410-893923546566\") " pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:38:57.064101 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:57.064098 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/65a3f603-5eb5-442c-a410-893923546566-proxy-tls\") pod \"sequence-graph-4108d-594769977b-sx7fn\" (UID: \"65a3f603-5eb5-442c-a410-893923546566\") " pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:38:57.064289 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:38:57.064215 2576 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-4108d-serving-cert: secret "sequence-graph-4108d-serving-cert" not found Jul 28 17:38:57.064357 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:38:57.064295 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65a3f603-5eb5-442c-a410-893923546566-proxy-tls podName:65a3f603-5eb5-442c-a410-893923546566 nodeName:}" failed. No retries permitted until 2026-07-28 17:38:57.564272803 +0000 UTC m=+1314.936821381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/65a3f603-5eb5-442c-a410-893923546566-proxy-tls") pod "sequence-graph-4108d-594769977b-sx7fn" (UID: "65a3f603-5eb5-442c-a410-893923546566") : secret "sequence-graph-4108d-serving-cert" not found Jul 28 17:38:57.064758 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:57.064738 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65a3f603-5eb5-442c-a410-893923546566-openshift-service-ca-bundle\") pod \"sequence-graph-4108d-594769977b-sx7fn\" (UID: \"65a3f603-5eb5-442c-a410-893923546566\") " pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:38:57.567622 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:57.567578 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/65a3f603-5eb5-442c-a410-893923546566-proxy-tls\") pod \"sequence-graph-4108d-594769977b-sx7fn\" (UID: \"65a3f603-5eb5-442c-a410-893923546566\") " pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:38:57.570040 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:57.570016 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/65a3f603-5eb5-442c-a410-893923546566-proxy-tls\") pod \"sequence-graph-4108d-594769977b-sx7fn\" (UID: \"65a3f603-5eb5-442c-a410-893923546566\") " pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:38:57.700258 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:57.700229 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:38:57.832054 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:57.831957 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn"] Jul 28 17:38:57.834393 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:38:57.834365 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a3f603_5eb5_442c_a410_893923546566.slice/crio-bcf0f4968db14340e860340097afb68b714d2d6b393f74292832b68333504bc7 WatchSource:0}: Error finding container bcf0f4968db14340e860340097afb68b714d2d6b393f74292832b68333504bc7: Status 404 returned error can't find the container with id bcf0f4968db14340e860340097afb68b714d2d6b393f74292832b68333504bc7 Jul 28 17:38:58.680563 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:58.680522 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" event={"ID":"65a3f603-5eb5-442c-a410-893923546566","Type":"ContainerStarted","Data":"1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2"} Jul 28 17:38:58.680824 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:58.680573 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" event={"ID":"65a3f603-5eb5-442c-a410-893923546566","Type":"ContainerStarted","Data":"bcf0f4968db14340e860340097afb68b714d2d6b393f74292832b68333504bc7"} Jul 28 17:38:58.680824 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:58.680604 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:38:58.697422 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:38:58.697374 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" podStartSLOduration=2.697359439 podStartE2EDuration="2.697359439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:38:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:38:58.696995906 +0000 UTC m=+1316.069544504" watchObservedRunningTime="2026-07-28 17:38:58.697359439 +0000 UTC m=+1316.069908037" Jul 28 17:39:04.690052 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:04.690021 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:39:06.895998 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:06.895968 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn"] Jul 28 17:39:06.896456 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:06.896234 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" podUID="65a3f603-5eb5-442c-a410-893923546566" containerName="sequence-graph-4108d" containerID="cri-o://1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2" gracePeriod=30 Jul 28 17:39:09.688924 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:09.688884 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" podUID="65a3f603-5eb5-442c-a410-893923546566" containerName="sequence-graph-4108d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:39:14.689138 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:14.689097 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" podUID="65a3f603-5eb5-442c-a410-893923546566" containerName="sequence-graph-4108d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:39:19.688824 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:19.688765 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" podUID="65a3f603-5eb5-442c-a410-893923546566" containerName="sequence-graph-4108d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:39:19.689275 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:19.688919 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:39:21.119943 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.119907 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92"] Jul 28 17:39:21.124554 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.124535 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:39:21.126968 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.126938 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-c57a1-kube-rbac-proxy-sar-config\"" Jul 28 17:39:21.127103 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.127084 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-c57a1-serving-cert\"" Jul 28 17:39:21.130979 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.130957 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92"] Jul 28 17:39:21.237118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.237092 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/632b065e-c3a9-4fa7-93aa-b2344af010f4-proxy-tls\") pod \"ensemble-graph-c57a1-76b899c654-jhp92\" (UID: \"632b065e-c3a9-4fa7-93aa-b2344af010f4\") " pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:39:21.237279 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.237139 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/632b065e-c3a9-4fa7-93aa-b2344af010f4-openshift-service-ca-bundle\") pod \"ensemble-graph-c57a1-76b899c654-jhp92\" (UID: \"632b065e-c3a9-4fa7-93aa-b2344af010f4\") " pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:39:21.338040 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.338005 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/632b065e-c3a9-4fa7-93aa-b2344af010f4-proxy-tls\") pod \"ensemble-graph-c57a1-76b899c654-jhp92\" (UID: \"632b065e-c3a9-4fa7-93aa-b2344af010f4\") " pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:39:21.338185 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.338051 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/632b065e-c3a9-4fa7-93aa-b2344af010f4-openshift-service-ca-bundle\") pod \"ensemble-graph-c57a1-76b899c654-jhp92\" (UID: \"632b065e-c3a9-4fa7-93aa-b2344af010f4\") " pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:39:21.338648 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.338625 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/632b065e-c3a9-4fa7-93aa-b2344af010f4-openshift-service-ca-bundle\") pod \"ensemble-graph-c57a1-76b899c654-jhp92\" (UID: \"632b065e-c3a9-4fa7-93aa-b2344af010f4\") " pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:39:21.340132 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.340109 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/632b065e-c3a9-4fa7-93aa-b2344af010f4-proxy-tls\") pod \"ensemble-graph-c57a1-76b899c654-jhp92\" (UID: \"632b065e-c3a9-4fa7-93aa-b2344af010f4\") " pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:39:21.436058 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.435996 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:39:21.576222 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.576193 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92"] Jul 28 17:39:21.578190 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:39:21.578162 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod632b065e_c3a9_4fa7_93aa_b2344af010f4.slice/crio-474765e815fd978a5ac09b18a37d3a5a2bad65b18b97b034b2ff9bd9e8a0a794 WatchSource:0}: Error finding container 474765e815fd978a5ac09b18a37d3a5a2bad65b18b97b034b2ff9bd9e8a0a794: Status 404 returned error can't find the container with id 474765e815fd978a5ac09b18a37d3a5a2bad65b18b97b034b2ff9bd9e8a0a794 Jul 28 17:39:21.786227 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.786193 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" event={"ID":"632b065e-c3a9-4fa7-93aa-b2344af010f4","Type":"ContainerStarted","Data":"890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3"} Jul 28 17:39:21.786446 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.786248 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" event={"ID":"632b065e-c3a9-4fa7-93aa-b2344af010f4","Type":"ContainerStarted","Data":"474765e815fd978a5ac09b18a37d3a5a2bad65b18b97b034b2ff9bd9e8a0a794"} Jul 28 17:39:21.786446 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.786323 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:39:21.808618 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:21.808574 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" podStartSLOduration=0.808559129 podStartE2EDuration="808.559129ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:39:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:39:21.806885834 +0000 UTC m=+1339.179434430" watchObservedRunningTime="2026-07-28 17:39:21.808559129 +0000 UTC m=+1339.181107725" Jul 28 17:39:24.688710 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:24.688667 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" podUID="65a3f603-5eb5-442c-a410-893923546566" containerName="sequence-graph-4108d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:39:27.796470 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:27.796392 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:39:29.688923 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:29.688888 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" podUID="65a3f603-5eb5-442c-a410-893923546566" containerName="sequence-graph-4108d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:39:34.688914 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:34.688862 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" podUID="65a3f603-5eb5-442c-a410-893923546566" containerName="sequence-graph-4108d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:39:37.042884 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.042858 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:39:37.155226 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.155157 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65a3f603-5eb5-442c-a410-893923546566-openshift-service-ca-bundle\") pod \"65a3f603-5eb5-442c-a410-893923546566\" (UID: \"65a3f603-5eb5-442c-a410-893923546566\") " Jul 28 17:39:37.155226 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.155221 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/65a3f603-5eb5-442c-a410-893923546566-proxy-tls\") pod \"65a3f603-5eb5-442c-a410-893923546566\" (UID: \"65a3f603-5eb5-442c-a410-893923546566\") " Jul 28 17:39:37.155486 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.155463 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65a3f603-5eb5-442c-a410-893923546566-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "65a3f603-5eb5-442c-a410-893923546566" (UID: "65a3f603-5eb5-442c-a410-893923546566"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:39:37.157223 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.157196 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65a3f603-5eb5-442c-a410-893923546566-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "65a3f603-5eb5-442c-a410-893923546566" (UID: "65a3f603-5eb5-442c-a410-893923546566"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:39:37.256721 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.256683 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65a3f603-5eb5-442c-a410-893923546566-openshift-service-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:39:37.256721 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.256712 2576 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/65a3f603-5eb5-442c-a410-893923546566-proxy-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:39:37.860205 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.860171 2576 generic.go:356] "Generic (PLEG): container finished" podID="65a3f603-5eb5-442c-a410-893923546566" containerID="1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2" exitCode=0 Jul 28 17:39:37.860406 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.860218 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" Jul 28 17:39:37.860406 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.860253 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" event={"ID":"65a3f603-5eb5-442c-a410-893923546566","Type":"ContainerDied","Data":"1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2"} Jul 28 17:39:37.860406 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.860301 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn" event={"ID":"65a3f603-5eb5-442c-a410-893923546566","Type":"ContainerDied","Data":"bcf0f4968db14340e860340097afb68b714d2d6b393f74292832b68333504bc7"} Jul 28 17:39:37.860406 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.860323 2576 scope.go:117] "RemoveContainer" containerID="1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2" Jul 28 17:39:37.868732 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.868718 2576 scope.go:117] "RemoveContainer" containerID="1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2" Jul 28 17:39:37.868986 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:39:37.868968 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2\": container with ID starting with 1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2 not found: ID does not exist" containerID="1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2" Jul 28 17:39:37.869026 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.868995 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2"} err="failed to get container status \"1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2\": rpc error: code = NotFound desc = could not find container \"1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2\": container with ID starting with 1f4472209f978ca357aa2cdec000433b23af4cb3bb12f15598c1ef499afe6fb2 not found: ID does not exist" Jul 28 17:39:37.881586 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.881562 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn"] Jul 28 17:39:37.889045 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:37.884375 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4108d-594769977b-sx7fn"] Jul 28 17:39:39.143690 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:39:39.143660 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65a3f603-5eb5-442c-a410-893923546566" path="/var/lib/kubelet/pods/65a3f603-5eb5-442c-a410-893923546566/volumes" Jul 28 17:40:07.154599 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.154561 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn"] Jul 28 17:40:07.155056 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.155039 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65a3f603-5eb5-442c-a410-893923546566" containerName="sequence-graph-4108d" Jul 28 17:40:07.155099 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.155059 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="65a3f603-5eb5-442c-a410-893923546566" containerName="sequence-graph-4108d" Jul 28 17:40:07.155146 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.155135 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="65a3f603-5eb5-442c-a410-893923546566" containerName="sequence-graph-4108d" Jul 28 17:40:07.159448 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.159428 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:40:07.161890 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.161863 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-c7add-serving-cert\"" Jul 28 17:40:07.162021 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.161927 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-c7add-kube-rbac-proxy-sar-config\"" Jul 28 17:40:07.169801 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.169748 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn"] Jul 28 17:40:07.278803 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.278744 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04236009-f9d3-4a65-911a-b7029f085638-proxy-tls\") pod \"sequence-graph-c7add-656869bfd7-9qnqn\" (UID: \"04236009-f9d3-4a65-911a-b7029f085638\") " pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:40:07.278970 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.278894 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04236009-f9d3-4a65-911a-b7029f085638-openshift-service-ca-bundle\") pod \"sequence-graph-c7add-656869bfd7-9qnqn\" (UID: \"04236009-f9d3-4a65-911a-b7029f085638\") " pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:40:07.379925 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.379892 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04236009-f9d3-4a65-911a-b7029f085638-proxy-tls\") pod \"sequence-graph-c7add-656869bfd7-9qnqn\" (UID: \"04236009-f9d3-4a65-911a-b7029f085638\") " pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:40:07.380077 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.379953 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04236009-f9d3-4a65-911a-b7029f085638-openshift-service-ca-bundle\") pod \"sequence-graph-c7add-656869bfd7-9qnqn\" (UID: \"04236009-f9d3-4a65-911a-b7029f085638\") " pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:40:07.380077 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:40:07.380031 2576 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-c7add-serving-cert: secret "sequence-graph-c7add-serving-cert" not found Jul 28 17:40:07.380147 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:40:07.380106 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04236009-f9d3-4a65-911a-b7029f085638-proxy-tls podName:04236009-f9d3-4a65-911a-b7029f085638 nodeName:}" failed. No retries permitted until 2026-07-28 17:40:07.880090289 +0000 UTC m=+1385.252638862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/04236009-f9d3-4a65-911a-b7029f085638-proxy-tls") pod "sequence-graph-c7add-656869bfd7-9qnqn" (UID: "04236009-f9d3-4a65-911a-b7029f085638") : secret "sequence-graph-c7add-serving-cert" not found Jul 28 17:40:07.380555 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.380538 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04236009-f9d3-4a65-911a-b7029f085638-openshift-service-ca-bundle\") pod \"sequence-graph-c7add-656869bfd7-9qnqn\" (UID: \"04236009-f9d3-4a65-911a-b7029f085638\") " pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:40:07.884254 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.884221 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04236009-f9d3-4a65-911a-b7029f085638-proxy-tls\") pod \"sequence-graph-c7add-656869bfd7-9qnqn\" (UID: \"04236009-f9d3-4a65-911a-b7029f085638\") " pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:40:07.886606 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:07.886583 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04236009-f9d3-4a65-911a-b7029f085638-proxy-tls\") pod \"sequence-graph-c7add-656869bfd7-9qnqn\" (UID: \"04236009-f9d3-4a65-911a-b7029f085638\") " pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:40:08.078077 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:08.078043 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:40:08.208187 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:08.208148 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn"] Jul 28 17:40:08.210449 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:40:08.210408 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04236009_f9d3_4a65_911a_b7029f085638.slice/crio-5a6ef29e25466f7ded68ab071ce49f15c93f9251013c7b1eb5d638fc5584653f WatchSource:0}: Error finding container 5a6ef29e25466f7ded68ab071ce49f15c93f9251013c7b1eb5d638fc5584653f: Status 404 returned error can't find the container with id 5a6ef29e25466f7ded68ab071ce49f15c93f9251013c7b1eb5d638fc5584653f Jul 28 17:40:08.999548 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:08.999512 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" event={"ID":"04236009-f9d3-4a65-911a-b7029f085638","Type":"ContainerStarted","Data":"0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df"} Jul 28 17:40:08.999727 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:08.999564 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" event={"ID":"04236009-f9d3-4a65-911a-b7029f085638","Type":"ContainerStarted","Data":"5a6ef29e25466f7ded68ab071ce49f15c93f9251013c7b1eb5d638fc5584653f"} Jul 28 17:40:08.999777 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:08.999740 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:40:09.016669 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:09.016624 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" podStartSLOduration=2.016608403 podStartE2EDuration="2.016608403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:40:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:40:09.01518494 +0000 UTC m=+1386.387733536" watchObservedRunningTime="2026-07-28 17:40:09.016608403 +0000 UTC m=+1386.389157011" Jul 28 17:40:15.010258 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:40:15.010227 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:42:03.121495 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:42:03.121462 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:42:03.122526 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:42:03.122506 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:47:03.146443 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:47:03.146417 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:47:03.147880 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:47:03.147861 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:47:35.820846 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:47:35.820816 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92"] Jul 28 17:47:35.821341 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:47:35.821011 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" podUID="632b065e-c3a9-4fa7-93aa-b2344af010f4" containerName="ensemble-graph-c57a1" containerID="cri-o://890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3" gracePeriod=30 Jul 28 17:47:37.794666 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:47:37.794625 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" podUID="632b065e-c3a9-4fa7-93aa-b2344af010f4" containerName="ensemble-graph-c57a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:47:42.794874 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:47:42.794836 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" podUID="632b065e-c3a9-4fa7-93aa-b2344af010f4" containerName="ensemble-graph-c57a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:47:47.794196 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:47:47.794157 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" podUID="632b065e-c3a9-4fa7-93aa-b2344af010f4" containerName="ensemble-graph-c57a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:47:47.794569 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:47:47.794250 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:47:52.794137 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:47:52.794093 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" podUID="632b065e-c3a9-4fa7-93aa-b2344af010f4" containerName="ensemble-graph-c57a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:47:57.794104 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:47:57.794060 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" podUID="632b065e-c3a9-4fa7-93aa-b2344af010f4" containerName="ensemble-graph-c57a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:48:02.794525 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:02.794481 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" podUID="632b065e-c3a9-4fa7-93aa-b2344af010f4" containerName="ensemble-graph-c57a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:48:05.963954 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:05.963931 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:48:06.017122 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.017091 2576 generic.go:356] "Generic (PLEG): container finished" podID="632b065e-c3a9-4fa7-93aa-b2344af010f4" containerID="890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3" exitCode=0 Jul 28 17:48:06.017220 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.017134 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" Jul 28 17:48:06.017220 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.017140 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" event={"ID":"632b065e-c3a9-4fa7-93aa-b2344af010f4","Type":"ContainerDied","Data":"890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3"} Jul 28 17:48:06.017297 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.017226 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92" event={"ID":"632b065e-c3a9-4fa7-93aa-b2344af010f4","Type":"ContainerDied","Data":"474765e815fd978a5ac09b18a37d3a5a2bad65b18b97b034b2ff9bd9e8a0a794"} Jul 28 17:48:06.017297 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.017255 2576 scope.go:117] "RemoveContainer" containerID="890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3" Jul 28 17:48:06.028579 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.028548 2576 scope.go:117] "RemoveContainer" containerID="890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3" Jul 28 17:48:06.028876 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:48:06.028856 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3\": container with ID starting with 890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3 not found: ID does not exist" containerID="890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3" Jul 28 17:48:06.028958 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.028884 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3"} err="failed to get container status \"890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3\": rpc error: code = NotFound desc = could not find container \"890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3\": container with ID starting with 890a739fcef4ca26f1f1db78301edbbc264b1cd115dbf461dfdf7a960c51dbd3 not found: ID does not exist" Jul 28 17:48:06.121272 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.121224 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/632b065e-c3a9-4fa7-93aa-b2344af010f4-openshift-service-ca-bundle\") pod \"632b065e-c3a9-4fa7-93aa-b2344af010f4\" (UID: \"632b065e-c3a9-4fa7-93aa-b2344af010f4\") " Jul 28 17:48:06.121363 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.121297 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/632b065e-c3a9-4fa7-93aa-b2344af010f4-proxy-tls\") pod \"632b065e-c3a9-4fa7-93aa-b2344af010f4\" (UID: \"632b065e-c3a9-4fa7-93aa-b2344af010f4\") " Jul 28 17:48:06.121598 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.121572 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/632b065e-c3a9-4fa7-93aa-b2344af010f4-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "632b065e-c3a9-4fa7-93aa-b2344af010f4" (UID: "632b065e-c3a9-4fa7-93aa-b2344af010f4"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:48:06.123180 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.123157 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632b065e-c3a9-4fa7-93aa-b2344af010f4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "632b065e-c3a9-4fa7-93aa-b2344af010f4" (UID: "632b065e-c3a9-4fa7-93aa-b2344af010f4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:48:06.222503 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.222470 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/632b065e-c3a9-4fa7-93aa-b2344af010f4-openshift-service-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:48:06.222503 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.222497 2576 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/632b065e-c3a9-4fa7-93aa-b2344af010f4-proxy-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:48:06.339645 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.339617 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92"] Jul 28 17:48:06.343650 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:06.343628 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c57a1-76b899c654-jhp92"] Jul 28 17:48:07.143339 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:07.143306 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="632b065e-c3a9-4fa7-93aa-b2344af010f4" path="/var/lib/kubelet/pods/632b065e-c3a9-4fa7-93aa-b2344af010f4/volumes" Jul 28 17:48:21.677958 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:21.677921 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn"] Jul 28 17:48:21.678348 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:21.678121 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" podUID="04236009-f9d3-4a65-911a-b7029f085638" containerName="sequence-graph-c7add" containerID="cri-o://0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df" gracePeriod=30 Jul 28 17:48:25.008401 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:25.008343 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" podUID="04236009-f9d3-4a65-911a-b7029f085638" containerName="sequence-graph-c7add" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:48:30.008056 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:30.007976 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" podUID="04236009-f9d3-4a65-911a-b7029f085638" containerName="sequence-graph-c7add" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:48:35.008906 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:35.008861 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" podUID="04236009-f9d3-4a65-911a-b7029f085638" containerName="sequence-graph-c7add" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:48:35.009403 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:35.008983 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:48:36.070966 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.070929 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4"] Jul 28 17:48:36.071429 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.071228 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="632b065e-c3a9-4fa7-93aa-b2344af010f4" containerName="ensemble-graph-c57a1" Jul 28 17:48:36.071429 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.071240 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="632b065e-c3a9-4fa7-93aa-b2344af010f4" containerName="ensemble-graph-c57a1" Jul 28 17:48:36.071429 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.071296 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="632b065e-c3a9-4fa7-93aa-b2344af010f4" containerName="ensemble-graph-c57a1" Jul 28 17:48:36.074040 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.074023 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:48:36.076412 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.076392 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-32bdd-kube-rbac-proxy-sar-config\"" Jul 28 17:48:36.076515 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.076393 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-32bdd-serving-cert\"" Jul 28 17:48:36.087282 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.084739 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4"] Jul 28 17:48:36.133559 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.133534 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b5c3262-e419-4778-9fa9-41c71ccc983f-openshift-service-ca-bundle\") pod \"splitter-graph-32bdd-ff74654bc-bvqc4\" (UID: \"1b5c3262-e419-4778-9fa9-41c71ccc983f\") " pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:48:36.133680 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.133580 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b5c3262-e419-4778-9fa9-41c71ccc983f-proxy-tls\") pod \"splitter-graph-32bdd-ff74654bc-bvqc4\" (UID: \"1b5c3262-e419-4778-9fa9-41c71ccc983f\") " pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:48:36.234389 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.234361 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b5c3262-e419-4778-9fa9-41c71ccc983f-openshift-service-ca-bundle\") pod \"splitter-graph-32bdd-ff74654bc-bvqc4\" (UID: \"1b5c3262-e419-4778-9fa9-41c71ccc983f\") " pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:48:36.234494 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.234405 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b5c3262-e419-4778-9fa9-41c71ccc983f-proxy-tls\") pod \"splitter-graph-32bdd-ff74654bc-bvqc4\" (UID: \"1b5c3262-e419-4778-9fa9-41c71ccc983f\") " pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:48:36.234973 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.234949 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b5c3262-e419-4778-9fa9-41c71ccc983f-openshift-service-ca-bundle\") pod \"splitter-graph-32bdd-ff74654bc-bvqc4\" (UID: \"1b5c3262-e419-4778-9fa9-41c71ccc983f\") " pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:48:36.236661 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.236635 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b5c3262-e419-4778-9fa9-41c71ccc983f-proxy-tls\") pod \"splitter-graph-32bdd-ff74654bc-bvqc4\" (UID: \"1b5c3262-e419-4778-9fa9-41c71ccc983f\") " pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:48:36.391587 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.391537 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:48:36.518989 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:36.518960 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4"] Jul 28 17:48:36.521569 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:48:36.521543 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5c3262_e419_4778_9fa9_41c71ccc983f.slice/crio-c20d96b76d30d190ebb3a8df5e9aa9b09d29641703a416789786817663f8e271 WatchSource:0}: Error finding container c20d96b76d30d190ebb3a8df5e9aa9b09d29641703a416789786817663f8e271: Status 404 returned error can't find the container with id c20d96b76d30d190ebb3a8df5e9aa9b09d29641703a416789786817663f8e271 Jul 28 17:48:37.154661 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:37.154624 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" event={"ID":"1b5c3262-e419-4778-9fa9-41c71ccc983f","Type":"ContainerStarted","Data":"f2e2816571cd7ccc67c8b5c8b36354af928635698db08692db271bb7e3bc3904"} Jul 28 17:48:37.155118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:37.154670 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" event={"ID":"1b5c3262-e419-4778-9fa9-41c71ccc983f","Type":"ContainerStarted","Data":"c20d96b76d30d190ebb3a8df5e9aa9b09d29641703a416789786817663f8e271"} Jul 28 17:48:37.155118 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:37.154731 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:48:37.173126 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:37.173074 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" podStartSLOduration=1.173062077 podStartE2EDuration="1.173062077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:48:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:48:37.171846501 +0000 UTC m=+1894.544395095" watchObservedRunningTime="2026-07-28 17:48:37.173062077 +0000 UTC m=+1894.545610672" Jul 28 17:48:40.008288 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:40.008245 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" podUID="04236009-f9d3-4a65-911a-b7029f085638" containerName="sequence-graph-c7add" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:48:43.163753 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:43.163725 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:48:45.008076 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:45.008033 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" podUID="04236009-f9d3-4a65-911a-b7029f085638" containerName="sequence-graph-c7add" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:48:46.155408 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:46.155329 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4"] Jul 28 17:48:46.155741 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:46.155482 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" podUID="1b5c3262-e419-4778-9fa9-41c71ccc983f" containerName="splitter-graph-32bdd" containerID="cri-o://f2e2816571cd7ccc67c8b5c8b36354af928635698db08692db271bb7e3bc3904" gracePeriod=30 Jul 28 17:48:48.162234 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:48.162193 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" podUID="1b5c3262-e419-4778-9fa9-41c71ccc983f" containerName="splitter-graph-32bdd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:48:50.008397 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:50.008329 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" podUID="04236009-f9d3-4a65-911a-b7029f085638" containerName="sequence-graph-c7add" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:48:51.825460 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:51.825440 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:48:51.949149 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:51.949088 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04236009-f9d3-4a65-911a-b7029f085638-openshift-service-ca-bundle\") pod \"04236009-f9d3-4a65-911a-b7029f085638\" (UID: \"04236009-f9d3-4a65-911a-b7029f085638\") " Jul 28 17:48:51.949278 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:51.949190 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04236009-f9d3-4a65-911a-b7029f085638-proxy-tls\") pod \"04236009-f9d3-4a65-911a-b7029f085638\" (UID: \"04236009-f9d3-4a65-911a-b7029f085638\") " Jul 28 17:48:51.949448 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:51.949427 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04236009-f9d3-4a65-911a-b7029f085638-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "04236009-f9d3-4a65-911a-b7029f085638" (UID: "04236009-f9d3-4a65-911a-b7029f085638"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:48:51.951155 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:51.951129 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04236009-f9d3-4a65-911a-b7029f085638-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "04236009-f9d3-4a65-911a-b7029f085638" (UID: "04236009-f9d3-4a65-911a-b7029f085638"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:48:52.050545 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:52.050524 2576 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04236009-f9d3-4a65-911a-b7029f085638-proxy-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:48:52.050545 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:52.050543 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04236009-f9d3-4a65-911a-b7029f085638-openshift-service-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:48:52.224764 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:52.224678 2576 generic.go:356] "Generic (PLEG): container finished" podID="04236009-f9d3-4a65-911a-b7029f085638" containerID="0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df" exitCode=0 Jul 28 17:48:52.224764 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:52.224720 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" Jul 28 17:48:52.224944 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:52.224761 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" event={"ID":"04236009-f9d3-4a65-911a-b7029f085638","Type":"ContainerDied","Data":"0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df"} Jul 28 17:48:52.224944 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:52.224826 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn" event={"ID":"04236009-f9d3-4a65-911a-b7029f085638","Type":"ContainerDied","Data":"5a6ef29e25466f7ded68ab071ce49f15c93f9251013c7b1eb5d638fc5584653f"} Jul 28 17:48:52.224944 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:52.224852 2576 scope.go:117] "RemoveContainer" containerID="0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df" Jul 28 17:48:52.234758 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:52.234741 2576 scope.go:117] "RemoveContainer" containerID="0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df" Jul 28 17:48:52.235021 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:48:52.235003 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df\": container with ID starting with 0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df not found: ID does not exist" containerID="0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df" Jul 28 17:48:52.235078 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:52.235029 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df"} err="failed to get container status \"0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df\": rpc error: code = NotFound desc = could not find container \"0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df\": container with ID starting with 0e83d118da5dea322b6f9bd910c6683f8a18903da442a26526c9a1af52be50df not found: ID does not exist" Jul 28 17:48:52.253881 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:52.253859 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn"] Jul 28 17:48:52.255647 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:52.255626 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c7add-656869bfd7-9qnqn"] Jul 28 17:48:53.143663 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:53.143633 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04236009-f9d3-4a65-911a-b7029f085638" path="/var/lib/kubelet/pods/04236009-f9d3-4a65-911a-b7029f085638/volumes" Jul 28 17:48:53.162454 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:53.162428 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" podUID="1b5c3262-e419-4778-9fa9-41c71ccc983f" containerName="splitter-graph-32bdd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:48:58.162695 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:58.162660 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" podUID="1b5c3262-e419-4778-9fa9-41c71ccc983f" containerName="splitter-graph-32bdd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:48:58.163075 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:48:58.162754 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:49:03.162445 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:03.162409 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" podUID="1b5c3262-e419-4778-9fa9-41c71ccc983f" containerName="splitter-graph-32bdd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:49:08.162518 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:08.162475 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" podUID="1b5c3262-e419-4778-9fa9-41c71ccc983f" containerName="splitter-graph-32bdd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:49:13.162871 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:13.162835 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" podUID="1b5c3262-e419-4778-9fa9-41c71ccc983f" containerName="splitter-graph-32bdd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:49:16.325502 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:16.325473 2576 generic.go:356] "Generic (PLEG): container finished" podID="1b5c3262-e419-4778-9fa9-41c71ccc983f" containerID="f2e2816571cd7ccc67c8b5c8b36354af928635698db08692db271bb7e3bc3904" exitCode=137 Jul 28 17:49:16.325876 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:16.325525 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" event={"ID":"1b5c3262-e419-4778-9fa9-41c71ccc983f","Type":"ContainerDied","Data":"f2e2816571cd7ccc67c8b5c8b36354af928635698db08692db271bb7e3bc3904"} Jul 28 17:49:16.325876 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:16.325577 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" event={"ID":"1b5c3262-e419-4778-9fa9-41c71ccc983f","Type":"ContainerDied","Data":"c20d96b76d30d190ebb3a8df5e9aa9b09d29641703a416789786817663f8e271"} Jul 28 17:49:16.325876 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:16.325593 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c20d96b76d30d190ebb3a8df5e9aa9b09d29641703a416789786817663f8e271" Jul 28 17:49:16.335355 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:16.335336 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:49:16.424999 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:16.424930 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b5c3262-e419-4778-9fa9-41c71ccc983f-openshift-service-ca-bundle\") pod \"1b5c3262-e419-4778-9fa9-41c71ccc983f\" (UID: \"1b5c3262-e419-4778-9fa9-41c71ccc983f\") " Jul 28 17:49:16.425119 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:16.425002 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b5c3262-e419-4778-9fa9-41c71ccc983f-proxy-tls\") pod \"1b5c3262-e419-4778-9fa9-41c71ccc983f\" (UID: \"1b5c3262-e419-4778-9fa9-41c71ccc983f\") " Jul 28 17:49:16.425350 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:16.425325 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b5c3262-e419-4778-9fa9-41c71ccc983f-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "1b5c3262-e419-4778-9fa9-41c71ccc983f" (UID: "1b5c3262-e419-4778-9fa9-41c71ccc983f"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:49:16.426853 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:16.426825 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b5c3262-e419-4778-9fa9-41c71ccc983f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1b5c3262-e419-4778-9fa9-41c71ccc983f" (UID: "1b5c3262-e419-4778-9fa9-41c71ccc983f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:49:16.526056 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:16.526025 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b5c3262-e419-4778-9fa9-41c71ccc983f-openshift-service-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:49:16.526056 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:16.526051 2576 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b5c3262-e419-4778-9fa9-41c71ccc983f-proxy-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:49:17.329824 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:17.329799 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4" Jul 28 17:49:17.348945 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:17.348918 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4"] Jul 28 17:49:17.355129 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:17.355107 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-32bdd-ff74654bc-bvqc4"] Jul 28 17:49:19.143516 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:19.143480 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b5c3262-e419-4778-9fa9-41c71ccc983f" path="/var/lib/kubelet/pods/1b5c3262-e419-4778-9fa9-41c71ccc983f/volumes" Jul 28 17:49:31.984616 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.984583 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc"] Jul 28 17:49:31.985187 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.985036 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04236009-f9d3-4a65-911a-b7029f085638" containerName="sequence-graph-c7add" Jul 28 17:49:31.985187 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.985056 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="04236009-f9d3-4a65-911a-b7029f085638" containerName="sequence-graph-c7add" Jul 28 17:49:31.985187 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.985087 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b5c3262-e419-4778-9fa9-41c71ccc983f" containerName="splitter-graph-32bdd" Jul 28 17:49:31.985187 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.985097 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b5c3262-e419-4778-9fa9-41c71ccc983f" containerName="splitter-graph-32bdd" Jul 28 17:49:31.985187 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.985180 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b5c3262-e419-4778-9fa9-41c71ccc983f" containerName="splitter-graph-32bdd" Jul 28 17:49:31.985439 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.985195 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="04236009-f9d3-4a65-911a-b7029f085638" containerName="sequence-graph-c7add" Jul 28 17:49:31.988492 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.988471 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 17:49:31.991966 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.991944 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 17:49:31.992081 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.991950 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tdp4m\"" Jul 28 17:49:31.992081 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.992008 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-c0ed1-serving-cert\"" Jul 28 17:49:31.994344 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:31.994324 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-c0ed1-kube-rbac-proxy-sar-config\"" Jul 28 17:49:32.000357 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:32.000335 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc"] Jul 28 17:49:32.138765 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:32.138732 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b21040c9-0cbf-4956-8112-4823c63772b3-proxy-tls\") pod \"switch-graph-c0ed1-695854fc69-pt6qc\" (UID: \"b21040c9-0cbf-4956-8112-4823c63772b3\") " pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 17:49:32.138925 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:32.138775 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b21040c9-0cbf-4956-8112-4823c63772b3-openshift-service-ca-bundle\") pod \"switch-graph-c0ed1-695854fc69-pt6qc\" (UID: \"b21040c9-0cbf-4956-8112-4823c63772b3\") " pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 17:49:32.239690 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:32.239614 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b21040c9-0cbf-4956-8112-4823c63772b3-proxy-tls\") pod \"switch-graph-c0ed1-695854fc69-pt6qc\" (UID: \"b21040c9-0cbf-4956-8112-4823c63772b3\") " pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 17:49:32.239833 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:32.239710 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b21040c9-0cbf-4956-8112-4823c63772b3-openshift-service-ca-bundle\") pod \"switch-graph-c0ed1-695854fc69-pt6qc\" (UID: \"b21040c9-0cbf-4956-8112-4823c63772b3\") " pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 17:49:32.239833 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:49:32.239746 2576 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-c0ed1-serving-cert: secret "switch-graph-c0ed1-serving-cert" not found Jul 28 17:49:32.239959 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:49:32.239849 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b21040c9-0cbf-4956-8112-4823c63772b3-proxy-tls podName:b21040c9-0cbf-4956-8112-4823c63772b3 nodeName:}" failed. No retries permitted until 2026-07-28 17:49:32.739828147 +0000 UTC m=+1950.112376727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b21040c9-0cbf-4956-8112-4823c63772b3-proxy-tls") pod "switch-graph-c0ed1-695854fc69-pt6qc" (UID: "b21040c9-0cbf-4956-8112-4823c63772b3") : secret "switch-graph-c0ed1-serving-cert" not found Jul 28 17:49:32.240413 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:32.240392 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b21040c9-0cbf-4956-8112-4823c63772b3-openshift-service-ca-bundle\") pod \"switch-graph-c0ed1-695854fc69-pt6qc\" (UID: \"b21040c9-0cbf-4956-8112-4823c63772b3\") " pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 17:49:32.744817 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:32.744766 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b21040c9-0cbf-4956-8112-4823c63772b3-proxy-tls\") pod \"switch-graph-c0ed1-695854fc69-pt6qc\" (UID: \"b21040c9-0cbf-4956-8112-4823c63772b3\") " pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 17:49:32.747087 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:32.747064 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b21040c9-0cbf-4956-8112-4823c63772b3-proxy-tls\") pod \"switch-graph-c0ed1-695854fc69-pt6qc\" (UID: \"b21040c9-0cbf-4956-8112-4823c63772b3\") " pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 17:49:32.899078 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:32.899055 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 17:49:33.032535 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:33.032478 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc"] Jul 28 17:49:33.034975 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:49:33.034947 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb21040c9_0cbf_4956_8112_4823c63772b3.slice/crio-e35240379c019b8f4848080af0662b1263bd5194f43a0b3d019469aaa1ce8d0d WatchSource:0}: Error finding container e35240379c019b8f4848080af0662b1263bd5194f43a0b3d019469aaa1ce8d0d: Status 404 returned error can't find the container with id e35240379c019b8f4848080af0662b1263bd5194f43a0b3d019469aaa1ce8d0d Jul 28 17:49:33.399994 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:33.399907 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" event={"ID":"b21040c9-0cbf-4956-8112-4823c63772b3","Type":"ContainerStarted","Data":"21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04"} Jul 28 17:49:33.399994 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:33.399972 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" event={"ID":"b21040c9-0cbf-4956-8112-4823c63772b3","Type":"ContainerStarted","Data":"e35240379c019b8f4848080af0662b1263bd5194f43a0b3d019469aaa1ce8d0d"} Jul 28 17:49:33.400156 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:33.400048 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 17:49:33.421462 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:33.421417 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" podStartSLOduration=2.421403898 podStartE2EDuration="2.421403898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:49:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:49:33.420719604 +0000 UTC m=+1950.793268199" watchObservedRunningTime="2026-07-28 17:49:33.421403898 +0000 UTC m=+1950.793952494" Jul 28 17:49:39.409423 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:39.409395 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 17:49:56.424809 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.424717 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v"] Jul 28 17:49:56.428121 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.428105 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:49:56.430643 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.430620 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-80e89-kube-rbac-proxy-sar-config\"" Jul 28 17:49:56.430643 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.430641 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-80e89-serving-cert\"" Jul 28 17:49:56.436675 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.436653 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v"] Jul 28 17:49:56.521747 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.521718 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab0078cf-f72f-43a9-a7e2-7e856296655e-openshift-service-ca-bundle\") pod \"splitter-graph-80e89-5f679bbbf7-v744v\" (UID: \"ab0078cf-f72f-43a9-a7e2-7e856296655e\") " pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:49:56.521914 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.521770 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ab0078cf-f72f-43a9-a7e2-7e856296655e-proxy-tls\") pod \"splitter-graph-80e89-5f679bbbf7-v744v\" (UID: \"ab0078cf-f72f-43a9-a7e2-7e856296655e\") " pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:49:56.622955 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.622926 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab0078cf-f72f-43a9-a7e2-7e856296655e-openshift-service-ca-bundle\") pod \"splitter-graph-80e89-5f679bbbf7-v744v\" (UID: \"ab0078cf-f72f-43a9-a7e2-7e856296655e\") " pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:49:56.623109 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.622966 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ab0078cf-f72f-43a9-a7e2-7e856296655e-proxy-tls\") pod \"splitter-graph-80e89-5f679bbbf7-v744v\" (UID: \"ab0078cf-f72f-43a9-a7e2-7e856296655e\") " pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:49:56.623606 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.623586 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab0078cf-f72f-43a9-a7e2-7e856296655e-openshift-service-ca-bundle\") pod \"splitter-graph-80e89-5f679bbbf7-v744v\" (UID: \"ab0078cf-f72f-43a9-a7e2-7e856296655e\") " pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:49:56.625239 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.625218 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ab0078cf-f72f-43a9-a7e2-7e856296655e-proxy-tls\") pod \"splitter-graph-80e89-5f679bbbf7-v744v\" (UID: \"ab0078cf-f72f-43a9-a7e2-7e856296655e\") " pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:49:56.738751 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.738653 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:49:56.871330 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:56.871299 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v"] Jul 28 17:49:56.873048 ip-10-0-136-215 kubenswrapper[2576]: W0728 17:49:56.873023 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab0078cf_f72f_43a9_a7e2_7e856296655e.slice/crio-c9f333d09691fad5877c7132e6e76640bf1eb6a8cb6de4466fc9eae78ee8fccb WatchSource:0}: Error finding container c9f333d09691fad5877c7132e6e76640bf1eb6a8cb6de4466fc9eae78ee8fccb: Status 404 returned error can't find the container with id c9f333d09691fad5877c7132e6e76640bf1eb6a8cb6de4466fc9eae78ee8fccb Jul 28 17:49:57.506962 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:57.506930 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" event={"ID":"ab0078cf-f72f-43a9-a7e2-7e856296655e","Type":"ContainerStarted","Data":"f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c"} Jul 28 17:49:57.507335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:57.507005 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" event={"ID":"ab0078cf-f72f-43a9-a7e2-7e856296655e","Type":"ContainerStarted","Data":"c9f333d09691fad5877c7132e6e76640bf1eb6a8cb6de4466fc9eae78ee8fccb"} Jul 28 17:49:57.507335 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:57.507133 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:49:57.525955 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:49:57.525916 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" podStartSLOduration=1.525903556 podStartE2EDuration="1.525903556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:49:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:49:57.525261009 +0000 UTC m=+1974.897809608" watchObservedRunningTime="2026-07-28 17:49:57.525903556 +0000 UTC m=+1974.898452150" Jul 28 17:50:03.516381 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:50:03.516338 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:52:03.172947 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:52:03.172913 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:52:03.175230 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:52:03.174601 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:55:03.261711 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:55:03.261677 2576 scope.go:117] "RemoveContainer" containerID="f2e2816571cd7ccc67c8b5c8b36354af928635698db08692db271bb7e3bc3904" Jul 28 17:57:03.196145 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:57:03.196038 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:57:03.200161 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:57:03.199170 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 17:58:11.093694 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:11.093656 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v"] Jul 28 17:58:11.094252 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:11.093928 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" podUID="ab0078cf-f72f-43a9-a7e2-7e856296655e" containerName="splitter-graph-80e89" containerID="cri-o://f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c" gracePeriod=30 Jul 28 17:58:13.514568 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:13.514530 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" podUID="ab0078cf-f72f-43a9-a7e2-7e856296655e" containerName="splitter-graph-80e89" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:58:18.514769 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:18.514730 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" podUID="ab0078cf-f72f-43a9-a7e2-7e856296655e" containerName="splitter-graph-80e89" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:58:23.514979 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:23.514941 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" podUID="ab0078cf-f72f-43a9-a7e2-7e856296655e" containerName="splitter-graph-80e89" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:58:23.515338 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:23.515036 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:58:28.515555 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:28.515512 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" podUID="ab0078cf-f72f-43a9-a7e2-7e856296655e" containerName="splitter-graph-80e89" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:58:33.515586 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:33.515552 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" podUID="ab0078cf-f72f-43a9-a7e2-7e856296655e" containerName="splitter-graph-80e89" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:58:38.515343 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:38.515306 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" podUID="ab0078cf-f72f-43a9-a7e2-7e856296655e" containerName="splitter-graph-80e89" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:58:41.241067 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.241042 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:58:41.406630 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.406551 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ab0078cf-f72f-43a9-a7e2-7e856296655e-proxy-tls\") pod \"ab0078cf-f72f-43a9-a7e2-7e856296655e\" (UID: \"ab0078cf-f72f-43a9-a7e2-7e856296655e\") " Jul 28 17:58:41.406630 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.406583 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab0078cf-f72f-43a9-a7e2-7e856296655e-openshift-service-ca-bundle\") pod \"ab0078cf-f72f-43a9-a7e2-7e856296655e\" (UID: \"ab0078cf-f72f-43a9-a7e2-7e856296655e\") " Jul 28 17:58:41.406994 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.406970 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab0078cf-f72f-43a9-a7e2-7e856296655e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "ab0078cf-f72f-43a9-a7e2-7e856296655e" (UID: "ab0078cf-f72f-43a9-a7e2-7e856296655e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:58:41.408458 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.408433 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab0078cf-f72f-43a9-a7e2-7e856296655e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ab0078cf-f72f-43a9-a7e2-7e856296655e" (UID: "ab0078cf-f72f-43a9-a7e2-7e856296655e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:58:41.507478 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.507454 2576 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ab0078cf-f72f-43a9-a7e2-7e856296655e-proxy-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:58:41.507478 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.507476 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab0078cf-f72f-43a9-a7e2-7e856296655e-openshift-service-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:58:41.751213 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.751139 2576 generic.go:356] "Generic (PLEG): container finished" podID="ab0078cf-f72f-43a9-a7e2-7e856296655e" containerID="f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c" exitCode=0 Jul 28 17:58:41.751213 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.751186 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" Jul 28 17:58:41.751369 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.751228 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" event={"ID":"ab0078cf-f72f-43a9-a7e2-7e856296655e","Type":"ContainerDied","Data":"f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c"} Jul 28 17:58:41.751369 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.751284 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v" event={"ID":"ab0078cf-f72f-43a9-a7e2-7e856296655e","Type":"ContainerDied","Data":"c9f333d09691fad5877c7132e6e76640bf1eb6a8cb6de4466fc9eae78ee8fccb"} Jul 28 17:58:41.751369 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.751314 2576 scope.go:117] "RemoveContainer" containerID="f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c" Jul 28 17:58:41.759936 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.759918 2576 scope.go:117] "RemoveContainer" containerID="f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c" Jul 28 17:58:41.760173 ip-10-0-136-215 kubenswrapper[2576]: E0728 17:58:41.760155 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c\": container with ID starting with f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c not found: ID does not exist" containerID="f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c" Jul 28 17:58:41.760220 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.760180 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c"} err="failed to get container status \"f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c\": rpc error: code = NotFound desc = could not find container \"f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c\": container with ID starting with f5c5d82bba114b41f383abb9a05f855aee45b674c62afa3fd41c174f4149765c not found: ID does not exist" Jul 28 17:58:41.774109 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.774087 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v"] Jul 28 17:58:41.781185 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:41.781165 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-80e89-5f679bbbf7-v744v"] Jul 28 17:58:43.143660 ip-10-0-136-215 kubenswrapper[2576]: I0728 17:58:43.143627 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab0078cf-f72f-43a9-a7e2-7e856296655e" path="/var/lib/kubelet/pods/ab0078cf-f72f-43a9-a7e2-7e856296655e/volumes" Jul 28 18:02:03.220377 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:02:03.220259 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 18:02:03.224285 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:02:03.223739 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 18:05:51.216707 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:05:51.216675 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc"] Jul 28 18:05:51.217284 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:05:51.216919 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" podUID="b21040c9-0cbf-4956-8112-4823c63772b3" containerName="switch-graph-c0ed1" containerID="cri-o://21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04" gracePeriod=30 Jul 28 18:05:54.407957 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:05:54.407921 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" podUID="b21040c9-0cbf-4956-8112-4823c63772b3" containerName="switch-graph-c0ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:05:59.408291 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:05:59.408241 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" podUID="b21040c9-0cbf-4956-8112-4823c63772b3" containerName="switch-graph-c0ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:06:04.408386 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:04.408338 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" podUID="b21040c9-0cbf-4956-8112-4823c63772b3" containerName="switch-graph-c0ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:06:04.408847 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:04.408467 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 18:06:07.875035 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:07.875006 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:08.798544 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:08.798513 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:09.407542 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:09.407506 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" podUID="b21040c9-0cbf-4956-8112-4823c63772b3" containerName="switch-graph-c0ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:06:09.685557 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:09.685481 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:10.585115 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:10.585089 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:11.493253 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:11.493223 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:12.353646 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:12.353617 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:13.211341 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:13.211314 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:14.084398 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:14.084364 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:14.407936 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:14.407849 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" podUID="b21040c9-0cbf-4956-8112-4823c63772b3" containerName="switch-graph-c0ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:06:14.926828 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:14.926777 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:15.805895 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:15.805865 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:16.686674 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:16.686642 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:17.570036 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:17.570009 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:18.443904 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:18.443869 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:19.389172 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:19.389146 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-c0ed1-695854fc69-pt6qc_b21040c9-0cbf-4956-8112-4823c63772b3/switch-graph-c0ed1/0.log" Jul 28 18:06:19.408424 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:19.408395 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" podUID="b21040c9-0cbf-4956-8112-4823c63772b3" containerName="switch-graph-c0ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:06:21.369689 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.369665 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 18:06:21.451239 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.451210 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b21040c9-0cbf-4956-8112-4823c63772b3-proxy-tls\") pod \"b21040c9-0cbf-4956-8112-4823c63772b3\" (UID: \"b21040c9-0cbf-4956-8112-4823c63772b3\") " Jul 28 18:06:21.451372 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.451264 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b21040c9-0cbf-4956-8112-4823c63772b3-openshift-service-ca-bundle\") pod \"b21040c9-0cbf-4956-8112-4823c63772b3\" (UID: \"b21040c9-0cbf-4956-8112-4823c63772b3\") " Jul 28 18:06:21.451608 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.451583 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b21040c9-0cbf-4956-8112-4823c63772b3-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b21040c9-0cbf-4956-8112-4823c63772b3" (UID: "b21040c9-0cbf-4956-8112-4823c63772b3"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 18:06:21.453260 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.453232 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b21040c9-0cbf-4956-8112-4823c63772b3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b21040c9-0cbf-4956-8112-4823c63772b3" (UID: "b21040c9-0cbf-4956-8112-4823c63772b3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 18:06:21.552601 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.552570 2576 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b21040c9-0cbf-4956-8112-4823c63772b3-proxy-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 18:06:21.552601 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.552598 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b21040c9-0cbf-4956-8112-4823c63772b3-openshift-service-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 18:06:21.683649 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.683619 2576 generic.go:356] "Generic (PLEG): container finished" podID="b21040c9-0cbf-4956-8112-4823c63772b3" containerID="21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04" exitCode=0 Jul 28 18:06:21.683818 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.683672 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" Jul 28 18:06:21.683818 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.683712 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" event={"ID":"b21040c9-0cbf-4956-8112-4823c63772b3","Type":"ContainerDied","Data":"21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04"} Jul 28 18:06:21.683818 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.683771 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc" event={"ID":"b21040c9-0cbf-4956-8112-4823c63772b3","Type":"ContainerDied","Data":"e35240379c019b8f4848080af0662b1263bd5194f43a0b3d019469aaa1ce8d0d"} Jul 28 18:06:21.683818 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.683813 2576 scope.go:117] "RemoveContainer" containerID="21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04" Jul 28 18:06:21.692392 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.692376 2576 scope.go:117] "RemoveContainer" containerID="21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04" Jul 28 18:06:21.692616 ip-10-0-136-215 kubenswrapper[2576]: E0728 18:06:21.692599 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04\": container with ID starting with 21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04 not found: ID does not exist" containerID="21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04" Jul 28 18:06:21.692670 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.692629 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04"} err="failed to get container status \"21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04\": rpc error: code = NotFound desc = could not find container \"21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04\": container with ID starting with 21b408b767c7fd7a96a046f1d303f90c0975bd8a9701f9fa395a1d014a953d04 not found: ID does not exist" Jul 28 18:06:21.706137 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.706117 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc"] Jul 28 18:06:21.710168 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:21.710150 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-c0ed1-695854fc69-pt6qc"] Jul 28 18:06:23.143150 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:23.143109 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b21040c9-0cbf-4956-8112-4823c63772b3" path="/var/lib/kubelet/pods/b21040c9-0cbf-4956-8112-4823c63772b3/volumes" Jul 28 18:06:26.850123 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:26.850052 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tjmr6_6c71f50e-2184-425b-9d24-0e2c2aebcaac/global-pull-secret-syncer/0.log" Jul 28 18:06:27.029558 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:27.029531 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xtm4c_de1c69f2-7d48-4a1f-9c2f-bf144ec6b80d/konnectivity-agent/0.log" Jul 28 18:06:27.120490 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:27.120412 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-215.ec2.internal_4746d2d48c5a0af4bf6819c4fa3b9ae9/haproxy/0.log" Jul 28 18:06:31.174470 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:31.174437 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-sfqvm_5d7e904f-06b5-4dfb-ac37-a97b38ace547/cluster-monitoring-operator/0.log" Jul 28 18:06:31.346212 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:31.346167 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-m552v_8d216a33-5710-4428-b12b-f5e5b8d88e8e/monitoring-plugin/0.log" Jul 28 18:06:31.591866 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:31.591840 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h8dc4_d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3/node-exporter/0.log" Jul 28 18:06:31.622422 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:31.622400 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h8dc4_d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3/kube-rbac-proxy/0.log" Jul 28 18:06:31.658388 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:31.658370 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h8dc4_d52ec06f-e4b6-47a3-8cf7-4acffc56b4a3/init-textfile/0.log" Jul 28 18:06:31.698095 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:31.698072 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-swjs8_e6d0450c-1d9e-4553-9b0c-8541e66e47a9/kube-rbac-proxy-main/0.log" Jul 28 18:06:31.729777 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:31.729752 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-swjs8_e6d0450c-1d9e-4553-9b0c-8541e66e47a9/kube-rbac-proxy-self/0.log" Jul 28 18:06:31.761104 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:31.761088 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-swjs8_e6d0450c-1d9e-4553-9b0c-8541e66e47a9/openshift-state-metrics/0.log" Jul 28 18:06:32.127059 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:32.127027 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-qpft2_76c12730-9f6e-451f-a485-db4456cf3e1d/prometheus-operator-admission-webhook/0.log" Jul 28 18:06:34.316869 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.316808 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/1.log" Jul 28 18:06:34.322818 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.322775 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-862sz_ae78b679-6ed6-4889-a920-585392ae4f8e/console-operator/2.log" Jul 28 18:06:34.877198 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.877166 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf"] Jul 28 18:06:34.877491 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.877479 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b21040c9-0cbf-4956-8112-4823c63772b3" containerName="switch-graph-c0ed1" Jul 28 18:06:34.877535 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.877493 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b21040c9-0cbf-4956-8112-4823c63772b3" containerName="switch-graph-c0ed1" Jul 28 18:06:34.877535 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.877505 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab0078cf-f72f-43a9-a7e2-7e856296655e" containerName="splitter-graph-80e89" Jul 28 18:06:34.877535 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.877511 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab0078cf-f72f-43a9-a7e2-7e856296655e" containerName="splitter-graph-80e89" Jul 28 18:06:34.877633 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.877567 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b21040c9-0cbf-4956-8112-4823c63772b3" containerName="switch-graph-c0ed1" Jul 28 18:06:34.877633 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.877576 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab0078cf-f72f-43a9-a7e2-7e856296655e" containerName="splitter-graph-80e89" Jul 28 18:06:34.881654 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.881637 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:34.884623 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.884601 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-92xdj\"/\"kube-root-ca.crt\"" Jul 28 18:06:34.885695 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.885677 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-92xdj\"/\"openshift-service-ca.crt\"" Jul 28 18:06:34.885819 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.885681 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-92xdj\"/\"default-dockercfg-5gwzn\"" Jul 28 18:06:34.892948 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.892928 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf"] Jul 28 18:06:34.949666 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.949643 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-sys\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:34.949772 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.949671 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-lib-modules\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:34.949772 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.949689 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-proc\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:34.949772 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.949708 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8h95\" (UniqueName: \"kubernetes.io/projected/a9ac4ddf-17e8-4862-817e-e6a133a1399f-kube-api-access-x8h95\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:34.949902 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:34.949848 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-podres\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.050962 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.050928 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-sys\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.050962 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.050964 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-lib-modules\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.051132 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.050984 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-proc\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.051132 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.051002 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8h95\" (UniqueName: \"kubernetes.io/projected/a9ac4ddf-17e8-4862-817e-e6a133a1399f-kube-api-access-x8h95\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.051132 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.051037 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-podres\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.051132 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.051041 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-sys\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.051132 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.051064 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-proc\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.051311 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.051129 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-lib-modules\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.051311 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.051140 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a9ac4ddf-17e8-4862-817e-e6a133a1399f-podres\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.059882 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.059857 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8h95\" (UniqueName: \"kubernetes.io/projected/a9ac4ddf-17e8-4862-817e-e6a133a1399f-kube-api-access-x8h95\") pod \"perf-node-gather-daemonset-p67kf\" (UID: \"a9ac4ddf-17e8-4862-817e-e6a133a1399f\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.191009 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.190956 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.319374 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.316676 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf"] Jul 28 18:06:35.320380 ip-10-0-136-215 kubenswrapper[2576]: W0728 18:06:35.320347 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda9ac4ddf_17e8_4862_817e_e6a133a1399f.slice/crio-c5190be58d465a96b97d43a0c76bc47624b35807a54cd9d7b63f785af2912192 WatchSource:0}: Error finding container c5190be58d465a96b97d43a0c76bc47624b35807a54cd9d7b63f785af2912192: Status 404 returned error can't find the container with id c5190be58d465a96b97d43a0c76bc47624b35807a54cd9d7b63f785af2912192 Jul 28 18:06:35.445847 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.445773 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-5msdh_ffa9c82e-220f-4b99-bf40-4136da9bd925/volume-data-source-validator/0.log" Jul 28 18:06:35.743043 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.742966 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" event={"ID":"a9ac4ddf-17e8-4862-817e-e6a133a1399f","Type":"ContainerStarted","Data":"3f64800e8d0d028223f329b7aabb3864e5d26e2b42c9c4499caec050168bf22f"} Jul 28 18:06:35.743043 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.743024 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" event={"ID":"a9ac4ddf-17e8-4862-817e-e6a133a1399f","Type":"ContainerStarted","Data":"c5190be58d465a96b97d43a0c76bc47624b35807a54cd9d7b63f785af2912192"} Jul 28 18:06:35.743209 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.743095 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:35.761165 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:35.761125 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" podStartSLOduration=1.761112152 podStartE2EDuration="1.761112152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:06:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:06:35.759547507 +0000 UTC m=+2973.132096103" watchObservedRunningTime="2026-07-28 18:06:35.761112152 +0000 UTC m=+2973.133660747" Jul 28 18:06:36.544717 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:36.544681 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tx79n_0a20f6b5-d596-481d-8de8-edb57100e326/dns/0.log" Jul 28 18:06:36.569135 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:36.569102 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tx79n_0a20f6b5-d596-481d-8de8-edb57100e326/kube-rbac-proxy/0.log" Jul 28 18:06:36.597470 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:36.597444 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4vvsd_09009b39-09ab-475e-bc10-1c0f559e4a07/dns-node-resolver/0.log" Jul 28 18:06:37.161110 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:37.161086 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-b7f4fb6c9-l874l_f128ad08-a030-47de-ae1b-18bbd74e505e/registry/0.log" Jul 28 18:06:37.186251 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:37.186225 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87p9h_c795a38b-13c3-4b44-ae37-9de0fd450a00/node-ca/0.log" Jul 28 18:06:38.550258 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:38.550230 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xg2v2_db7b2d75-c2fa-4f26-986e-ce1164ae5b67/serve-healthcheck-canary/0.log" Jul 28 18:06:38.997117 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:38.997039 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-fdb6z_eef7416b-d05b-4b67-b553-dc593460efb1/insights-operator/0.log" Jul 28 18:06:38.998033 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:38.998017 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-fdb6z_eef7416b-d05b-4b67-b553-dc593460efb1/insights-operator/1.log" Jul 28 18:06:39.025996 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:39.025956 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9cghg_eae07057-b8a0-4138-88fd-0ba713b04676/kube-rbac-proxy/0.log" Jul 28 18:06:39.070988 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:39.070970 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9cghg_eae07057-b8a0-4138-88fd-0ba713b04676/exporter/0.log" Jul 28 18:06:39.108161 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:39.108144 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9cghg_eae07057-b8a0-4138-88fd-0ba713b04676/extractor/0.log" Jul 28 18:06:41.656776 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:41.656745 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-84fc959947-7mvnz_e8de49ea-a46e-4a86-81fb-734718e97b4e/manager/0.log" Jul 28 18:06:41.679321 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:41.679299 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-n52gf_ed51d813-9bd8-4978-beb2-e65947646bbe/server/0.log" Jul 28 18:06:41.759007 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:41.758980 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-p67kf" Jul 28 18:06:48.581750 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:48.581718 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8f82_ed55c986-c239-4f25-a72f-539854fa246d/kube-multus-additional-cni-plugins/0.log" Jul 28 18:06:48.636802 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:48.636770 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8f82_ed55c986-c239-4f25-a72f-539854fa246d/egress-router-binary-copy/0.log" Jul 28 18:06:48.665756 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:48.665733 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8f82_ed55c986-c239-4f25-a72f-539854fa246d/cni-plugins/0.log" Jul 28 18:06:48.696068 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:48.696044 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8f82_ed55c986-c239-4f25-a72f-539854fa246d/bond-cni-plugin/0.log" Jul 28 18:06:48.724459 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:48.724434 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8f82_ed55c986-c239-4f25-a72f-539854fa246d/routeoverride-cni/0.log" Jul 28 18:06:48.749955 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:48.749928 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8f82_ed55c986-c239-4f25-a72f-539854fa246d/whereabouts-cni-bincopy/0.log" Jul 28 18:06:48.776838 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:48.776816 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8f82_ed55c986-c239-4f25-a72f-539854fa246d/whereabouts-cni/0.log" Jul 28 18:06:48.815758 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:48.815735 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-flxnq_bc2a32ca-505c-45a6-8ab0-8b994aa479f0/kube-multus/0.log" Jul 28 18:06:48.964124 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:48.964047 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s9qjh_e761eac4-98d4-494a-be69-82b2f0e6e4d4/network-metrics-daemon/0.log" Jul 28 18:06:48.987389 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:48.987366 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s9qjh_e761eac4-98d4-494a-be69-82b2f0e6e4d4/kube-rbac-proxy/0.log" Jul 28 18:06:49.892315 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:49.892274 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c29m8_54b22a64-9b05-4404-be84-b23e2c9d6af6/ovn-controller/0.log" Jul 28 18:06:49.914812 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:49.914777 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c29m8_54b22a64-9b05-4404-be84-b23e2c9d6af6/ovn-acl-logging/0.log" Jul 28 18:06:49.945505 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:49.945481 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c29m8_54b22a64-9b05-4404-be84-b23e2c9d6af6/kube-rbac-proxy-node/0.log" Jul 28 18:06:49.971519 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:49.971501 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c29m8_54b22a64-9b05-4404-be84-b23e2c9d6af6/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 18:06:49.997558 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:49.997537 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c29m8_54b22a64-9b05-4404-be84-b23e2c9d6af6/northd/0.log" Jul 28 18:06:50.024898 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:50.024878 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c29m8_54b22a64-9b05-4404-be84-b23e2c9d6af6/nbdb/0.log" Jul 28 18:06:50.054157 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:50.054134 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c29m8_54b22a64-9b05-4404-be84-b23e2c9d6af6/sbdb/0.log" Jul 28 18:06:50.107217 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:50.107199 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c29m8_54b22a64-9b05-4404-be84-b23e2c9d6af6/ovnkube-controller/0.log" Jul 28 18:06:51.832421 ip-10-0-136-215 kubenswrapper[2576]: I0728 18:06:51.832389 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-5g2mj_4701c1b9-f966-41c6-bd60-86d9fc2bf11e/network-check-target-container/0.log"