Jul 31 23:46:21.631845 ip-10-0-131-119 systemd[1]: Starting Kubernetes Kubelet... Jul 31 23:46:22.058129 ip-10-0-131-119 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 23:46:22.058129 ip-10-0-131-119 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 31 23:46:22.058129 ip-10-0-131-119 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 23:46:22.058129 ip-10-0-131-119 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 31 23:46:22.058129 ip-10-0-131-119 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 23:46:22.059108 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.059018 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 31 23:46:22.064715 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064693 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 23:46:22.064715 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064710 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 23:46:22.064715 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064714 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 23:46:22.064715 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064718 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 23:46:22.064715 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064722 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064726 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064729 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064733 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064736 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064739 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064741 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064744 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064747 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064749 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064752 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064755 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064757 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064760 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064762 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064768 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064771 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064773 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064776 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064778 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 31 23:46:22.064908 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064781 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064783 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064786 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064788 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064791 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064793 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064796 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064799 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064802 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064804 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064807 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064810 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064812 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064814 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064819 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064821 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064825 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064827 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064830 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064832 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 23:46:22.065393 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064835 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064838 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064840 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064843 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064845 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064849 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064854 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064857 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064860 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064862 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064865 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064868 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064870 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064873 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064875 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064878 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064880 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064883 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064885 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064887 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 23:46:22.065898 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064890 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064893 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064895 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064898 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064900 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064903 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064907 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064910 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064913 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064916 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064919 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064922 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064925 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064929 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064932 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064935 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064938 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064941 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064943 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 23:46:22.066386 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064947 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064949 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064952 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064954 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064957 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064959 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064962 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064964 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064967 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064970 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064973 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064976 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064978 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064983 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064987 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.064990 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065688 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065695 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065698 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 23:46:22.066857 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065701 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065704 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065707 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065710 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065712 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065715 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065717 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065720 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065723 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065725 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065728 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065731 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065734 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065737 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065740 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065742 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065745 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065748 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065750 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065754 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 23:46:22.067339 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065758 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065760 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065763 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065766 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065768 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065771 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065774 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065776 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065779 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065782 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065785 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065787 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065790 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065793 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065795 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065798 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065800 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065803 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065805 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065808 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 23:46:22.067863 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065810 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065813 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065815 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065818 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065821 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065824 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065827 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065830 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065833 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065836 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065838 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065841 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065844 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065846 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065849 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065852 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065854 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065857 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065860 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065862 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 23:46:22.068361 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065867 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065870 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065873 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065876 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065878 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065881 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065883 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065888 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065890 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065893 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065896 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065899 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065901 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065904 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065907 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065910 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065912 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065915 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065917 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065920 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 23:46:22.068871 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065923 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065926 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065929 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065931 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065936 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065939 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065942 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065945 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065948 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065950 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065953 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065957 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065961 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065965 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065967 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.065970 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066335 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066341 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066345 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 23:46:22.069362 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066349 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066352 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066355 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066358 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066362 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066365 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066367 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066370 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066374 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066377 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066379 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066382 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066385 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066389 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066392 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066395 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066398 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066401 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066403 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 23:46:22.069861 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066406 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066408 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066411 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066413 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066416 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066419 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066423 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066425 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066428 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066430 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066433 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066435 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066438 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066440 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066443 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066445 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066448 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066452 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066455 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066457 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 23:46:22.070328 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066460 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066462 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066465 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066468 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066470 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066472 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066475 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066478 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066480 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066483 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066485 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066487 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066490 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066493 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066495 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066497 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066500 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066502 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066505 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066508 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 23:46:22.070830 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066511 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066513 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066516 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066518 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066521 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066523 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066526 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066528 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066531 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066534 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066537 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066539 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066543 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066560 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066563 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066566 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066569 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066572 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066575 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 23:46:22.071349 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066577 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066581 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066584 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066588 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066592 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066596 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066598 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066601 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066605 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066607 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066610 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066613 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066616 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066618 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066621 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066624 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066626 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.066629 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066697 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066708 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 31 23:46:22.071825 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066714 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066718 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066725 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066729 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066733 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066737 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066740 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066743 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066747 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066750 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066753 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066756 2582 flags.go:64] FLAG: --cgroup-root="" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066759 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066762 2582 flags.go:64] FLAG: --client-ca-file="" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066766 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066768 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066774 2582 flags.go:64] FLAG: --cluster-domain="" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066776 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066779 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066782 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066786 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066790 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066793 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 31 23:46:22.072342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066796 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066799 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066802 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066805 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066808 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066811 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066814 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066817 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066820 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066823 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066826 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066830 2582 flags.go:64] FLAG: --enable-server="true" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066833 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066837 2582 flags.go:64] FLAG: --event-burst="100" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066840 2582 flags.go:64] FLAG: --event-qps="50" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066843 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066846 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066849 2582 flags.go:64] FLAG: --eviction-hard="" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066852 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066855 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066858 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066862 2582 flags.go:64] FLAG: --eviction-soft="" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066864 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066867 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066874 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 31 23:46:22.072916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066877 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066881 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066884 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066886 2582 flags.go:64] FLAG: --feature-gates="" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066890 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066893 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066897 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066900 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066903 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066906 2582 flags.go:64] FLAG: --help="false" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066909 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066912 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066915 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066927 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066930 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066934 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066937 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066940 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066943 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066946 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066949 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066952 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066955 2582 flags.go:64] FLAG: --kube-reserved="" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066958 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 31 23:46:22.073538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066961 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066964 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066966 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066969 2582 flags.go:64] FLAG: --lock-file="" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066972 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066975 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066979 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066985 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066988 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066991 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066994 2582 flags.go:64] FLAG: --logging-format="text" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.066997 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067001 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067004 2582 flags.go:64] FLAG: --manifest-url="" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067009 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067013 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067016 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067021 2582 flags.go:64] FLAG: --max-pods="110" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067024 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067027 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067030 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067033 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067036 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067038 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067041 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 31 23:46:22.074148 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067048 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067051 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067054 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067057 2582 flags.go:64] FLAG: --pod-cidr="" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067060 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067065 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067068 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067071 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067074 2582 flags.go:64] FLAG: --port="10250" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067077 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067080 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0557a4eb2c37096d6" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067084 2582 flags.go:64] FLAG: --qos-reserved="" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067087 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067090 2582 flags.go:64] FLAG: --register-node="true" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067092 2582 flags.go:64] FLAG: --register-with-taints="" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067096 2582 flags.go:64] FLAG: --registry-burst="10" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067099 2582 flags.go:64] FLAG: --registry-qps="5" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067102 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067104 2582 flags.go:64] FLAG: --reserved-memory="" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067108 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067111 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067114 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067118 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067121 2582 flags.go:64] FLAG: --runonce="false" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067124 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 31 23:46:22.074776 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067127 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067130 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067133 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067136 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067139 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067142 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067145 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067148 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067151 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067154 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067157 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067160 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067163 2582 flags.go:64] FLAG: --system-cgroups="" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067165 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067171 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067173 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067176 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067180 2582 flags.go:64] FLAG: --tls-min-version="" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067183 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067186 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067189 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067191 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067194 2582 flags.go:64] FLAG: --v="2" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067198 2582 flags.go:64] FLAG: --version="false" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067202 2582 flags.go:64] FLAG: --vmodule="" Jul 31 23:46:22.075382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067206 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.067209 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067333 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067337 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067342 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067345 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067349 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067352 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067355 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067358 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067360 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067363 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067366 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067369 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067371 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067374 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067376 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067379 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067381 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067384 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 23:46:22.076038 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067387 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067390 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067392 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067394 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067397 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067400 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067403 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067405 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067408 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067410 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067413 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067416 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067419 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067423 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067426 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067429 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067433 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067435 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067440 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 23:46:22.076575 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067442 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067445 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067447 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067450 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067452 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067455 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067458 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067460 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067463 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067465 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067468 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067470 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067473 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067476 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067478 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067482 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067486 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067489 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067492 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 23:46:22.077035 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067494 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067497 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067500 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067502 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067505 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067508 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067510 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067514 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067516 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067519 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067523 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067526 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067531 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067535 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067538 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067540 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067543 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067562 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067564 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067568 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 23:46:22.077501 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067570 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067573 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067575 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067578 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067580 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067583 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067585 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067588 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067590 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067593 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067595 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067598 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067601 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067603 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067606 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067609 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067612 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067614 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067617 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067619 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 23:46:22.078005 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067622 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067625 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.067630 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.068198 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.074859 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.074879 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.075664 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.075672 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075745 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075751 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075755 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075758 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075761 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075764 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075767 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 23:46:22.078485 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075770 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075773 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075775 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075778 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075781 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075783 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075786 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075788 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075791 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075794 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075796 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075799 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075801 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075804 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075806 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075809 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075812 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075815 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075817 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075820 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 23:46:22.078876 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075822 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075825 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075834 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075837 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075840 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075842 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075845 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075847 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075850 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075854 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075859 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075862 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075865 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075868 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075871 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075874 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075877 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075880 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075883 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 23:46:22.079372 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075886 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075888 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075891 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075893 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075896 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075899 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075901 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075904 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075907 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075909 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075912 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075914 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075917 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075920 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075923 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075927 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075934 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075937 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075940 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 23:46:22.079846 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075944 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075948 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075951 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075954 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075956 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075959 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075962 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075964 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075967 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075970 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075972 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075975 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075977 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075980 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075982 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075985 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075987 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075990 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075993 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075995 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 23:46:22.080314 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.075997 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076000 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076003 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076011 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076014 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076016 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076019 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076021 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076025 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076027 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076037 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076040 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076043 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076046 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.076052 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 23:46:22.080819 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076196 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076201 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076204 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076206 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076209 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076211 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076214 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076217 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076219 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076222 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076224 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076227 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076230 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076232 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076235 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076237 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076240 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076242 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076245 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 23:46:22.081194 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076247 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076251 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076255 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076258 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076261 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076264 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076267 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076269 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076272 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076280 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076283 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076285 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076288 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076291 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076293 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076296 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076298 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076301 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076303 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076306 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 23:46:22.081697 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076308 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076311 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076313 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076316 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076318 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076321 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076323 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076325 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076328 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076331 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076333 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076336 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076339 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076341 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076344 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076347 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076349 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076352 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076355 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076357 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 23:46:22.082180 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076360 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076362 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076370 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076373 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076376 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076380 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076383 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076386 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076389 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076391 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076394 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076396 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076399 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076401 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076404 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076406 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076409 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076411 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076414 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 23:46:22.082682 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076416 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076420 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076423 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076426 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076428 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076431 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076434 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076437 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076440 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076443 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076445 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076448 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076451 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076453 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076456 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076459 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076467 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076469 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 23:46:22.083150 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076472 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 23:46:22.083607 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076474 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 23:46:22.083607 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:22.076477 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 23:46:22.083607 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.076482 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 23:46:22.083607 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.076610 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 31 23:46:22.083607 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.078635 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 31 23:46:22.083607 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.080162 2582 server.go:1019] "Starting client certificate rotation" Jul 31 23:46:22.083607 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.080272 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 31 23:46:22.083607 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.080316 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 31 23:46:22.083607 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.083416 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 31 23:46:22.090214 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.090190 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 31 23:46:22.106282 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.106258 2582 log.go:25] "Validated CRI v1 runtime API" Jul 31 23:46:22.108342 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.108327 2582 log.go:25] "Validated CRI v1 image API" Jul 31 23:46:22.109145 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.109125 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 31 23:46:22.110294 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.110274 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 31 23:46:22.111561 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.111528 2582 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 93b526cc-acde-49ba-b586-3556d6bf304a:/dev/nvme0n1p4 d196da0b-3319-4deb-a295-9ac86d4ba70a:/dev/nvme0n1p3] Jul 31 23:46:22.111621 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.111566 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 31 23:46:22.117596 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.117482 2582 manager.go:216] Machine: {Timestamp:2026-07-31 23:46:22.115983763 +0000 UTC m=+0.361691916 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100218 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec296f4dff34d06be0bc5eee465dcb53 SystemUUID:ec296f4d-ff34-d06b-e0bc-5eee465dcb53 BootID:465de54b-1cc9-4410-913d-620775e27024 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:df:84:02:30:8d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:df:84:02:30:8d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:71:78:ac:f9:5f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 31 23:46:22.117596 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.117592 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 31 23:46:22.117702 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.117677 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 31 23:46:22.120990 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.120961 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 31 23:46:22.121134 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.120992 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-119.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 31 23:46:22.121730 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.121720 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 31 23:46:22.121766 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.121733 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 31 23:46:22.121766 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.121741 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 31 23:46:22.122608 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.122598 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 31 23:46:22.123402 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.123392 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 31 23:46:22.126222 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.126211 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 31 23:46:22.126341 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.126332 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 31 23:46:22.126373 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.126368 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 31 23:46:22.126400 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.126378 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 31 23:46:22.126400 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.126389 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 31 23:46:22.126460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.126408 2582 kubelet.go:398] "Adding apiserver pod source" Jul 31 23:46:22.126460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.126417 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 31 23:46:22.126972 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.126949 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 31 23:46:22.127100 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.127004 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 31 23:46:22.129580 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.129543 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-swvt7" Jul 31 23:46:22.129892 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.129878 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 31 23:46:22.131783 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.131760 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 31 23:46:22.132810 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.132747 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 31 23:46:22.134972 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.134952 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 31 23:46:22.135055 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.134977 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 31 23:46:22.135055 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.134984 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 31 23:46:22.135055 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.134989 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 31 23:46:22.135055 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.134999 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 31 23:46:22.135055 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.135005 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 31 23:46:22.135055 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.135013 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 31 23:46:22.135055 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.135018 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 31 23:46:22.135055 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.135026 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 31 23:46:22.135055 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.135032 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 31 23:46:22.135055 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.135040 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 31 23:46:22.135055 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.135045 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 31 23:46:22.135362 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.135013 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-swvt7" Jul 31 23:46:22.135832 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.135822 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 31 23:46:22.135832 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.135832 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 31 23:46:22.136852 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.136832 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-119.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 31 23:46:22.136933 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.136916 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 31 23:46:22.139033 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.139022 2582 server.go:1268] "Started kubelet" Jul 31 23:46:22.139113 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.139083 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 31 23:46:22.139178 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.139105 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 31 23:46:22.139178 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.139152 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 31 23:46:22.139874 ip-10-0-131-119 systemd[1]: Started Kubernetes Kubelet. Jul 31 23:46:22.140254 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.140241 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 31 23:46:22.141147 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.141130 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 31 23:46:22.146197 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.146175 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 31 23:46:22.146689 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.146676 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 31 23:46:22.146758 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.146686 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 31 23:46:22.147113 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.146739 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:22.147639 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.147623 2582 factory.go:55] Registering systemd factory Jul 31 23:46:22.147639 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.147641 2582 factory.go:223] Registration of the systemd container factory successfully Jul 31 23:46:22.147854 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.147843 2582 factory.go:153] Registering CRI-O factory Jul 31 23:46:22.147916 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.147858 2582 factory.go:223] Registration of the crio container factory successfully Jul 31 23:46:22.148001 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.147920 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 31 23:46:22.148001 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.147950 2582 factory.go:103] Registering Raw factory Jul 31 23:46:22.148001 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.147967 2582 manager.go:1196] Started watching for new ooms in manager Jul 31 23:46:22.148138 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.148124 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 31 23:46:22.148168 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.148126 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 31 23:46:22.148168 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.148152 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 31 23:46:22.148280 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.148265 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 31 23:46:22.148326 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.148281 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 31 23:46:22.148855 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.148838 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 23:46:22.149036 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.149022 2582 manager.go:349] Starting recovery of all containers Jul 31 23:46:22.154690 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.154669 2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-131-119.ec2.internal" not found Jul 31 23:46:22.156884 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.156848 2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-119.ec2.internal\" not found" node="ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.159898 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.159883 2582 manager.go:354] Recovery completed Jul 31 23:46:22.164191 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.164177 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 23:46:22.166336 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.166225 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientMemory" Jul 31 23:46:22.166336 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.166254 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 23:46:22.166336 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.166264 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientPID" Jul 31 23:46:22.167376 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.167361 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 31 23:46:22.167450 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.167379 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 31 23:46:22.167450 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.167364 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 23:46:22.167450 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.167395 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 31 23:46:22.168144 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.168127 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientMemory" Jul 31 23:46:22.168216 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.168160 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 23:46:22.168216 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.168174 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientPID" Jul 31 23:46:22.168494 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.168481 2582 policy_none.go:49] "None policy: Start" Jul 31 23:46:22.168602 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.168501 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 31 23:46:22.168602 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.168515 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 31 23:46:22.169700 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.169688 2582 policy_none.go:47] "Start" Jul 31 23:46:22.170473 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.170443 2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-131-119.ec2.internal" not found Jul 31 23:46:22.197509 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.197295 2582 manager.go:337] "Starting Device Plugin manager" Jul 31 23:46:22.213143 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.197535 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 31 23:46:22.213143 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.197564 2582 server.go:87] "Starting device plugin registration server" Jul 31 23:46:22.213143 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.197800 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 31 23:46:22.213143 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.197814 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 31 23:46:22.213143 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.197996 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 31 23:46:22.213143 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.198074 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 31 23:46:22.213143 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.198084 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 31 23:46:22.213143 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.198658 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 31 23:46:22.213143 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.198731 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:22.229664 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.229645 2582 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-131-119.ec2.internal" not found Jul 31 23:46:22.278201 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.278150 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 31 23:46:22.279503 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.279478 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 31 23:46:22.279656 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.279566 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 31 23:46:22.279656 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.279612 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 31 23:46:22.279656 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.279655 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 31 23:46:22.281872 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.281850 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 23:46:22.298592 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.298560 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 23:46:22.299432 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.299411 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientMemory" Jul 31 23:46:22.299538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.299452 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 23:46:22.299538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.299467 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientPID" Jul 31 23:46:22.299538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.299498 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.308453 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.308398 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.308453 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.308421 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-119.ec2.internal\": node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:22.321652 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.321626 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:22.380425 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.380369 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-119.ec2.internal"] Jul 31 23:46:22.380574 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.380468 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 23:46:22.382043 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.382025 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientMemory" Jul 31 23:46:22.382146 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.382060 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 23:46:22.382146 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.382077 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientPID" Jul 31 23:46:22.383512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.383495 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 23:46:22.383646 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.383630 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.383691 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.383664 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 23:46:22.384279 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.384264 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientMemory" Jul 31 23:46:22.384351 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.384291 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 23:46:22.384351 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.384303 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientPID" Jul 31 23:46:22.384351 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.384268 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientMemory" Jul 31 23:46:22.384454 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.384368 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 23:46:22.384454 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.384380 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientPID" Jul 31 23:46:22.385312 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.385298 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.385378 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.385323 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 23:46:22.386003 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.385988 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientMemory" Jul 31 23:46:22.386075 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.386011 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 23:46:22.386075 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.386024 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeHasSufficientPID" Jul 31 23:46:22.415357 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.415332 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-119.ec2.internal\" not found" node="ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.419883 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.419867 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-119.ec2.internal\" not found" node="ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.421908 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.421894 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:22.449587 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.449562 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de477c8056e974ae14ba31ef7a62870f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal\" (UID: \"de477c8056e974ae14ba31ef7a62870f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.449646 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.449593 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de477c8056e974ae14ba31ef7a62870f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal\" (UID: \"de477c8056e974ae14ba31ef7a62870f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.449646 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.449610 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/148fdac59207384bb5b95d23db64dc63-config\") pod \"kube-apiserver-proxy-ip-10-0-131-119.ec2.internal\" (UID: \"148fdac59207384bb5b95d23db64dc63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.522512 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.522489 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:22.549844 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.549819 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de477c8056e974ae14ba31ef7a62870f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal\" (UID: \"de477c8056e974ae14ba31ef7a62870f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.549892 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.549850 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de477c8056e974ae14ba31ef7a62870f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal\" (UID: \"de477c8056e974ae14ba31ef7a62870f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.549892 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.549878 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de477c8056e974ae14ba31ef7a62870f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal\" (UID: \"de477c8056e974ae14ba31ef7a62870f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.549962 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.549895 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de477c8056e974ae14ba31ef7a62870f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal\" (UID: \"de477c8056e974ae14ba31ef7a62870f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.549962 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.549914 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/148fdac59207384bb5b95d23db64dc63-config\") pod \"kube-apiserver-proxy-ip-10-0-131-119.ec2.internal\" (UID: \"148fdac59207384bb5b95d23db64dc63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.549962 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:22.549934 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/148fdac59207384bb5b95d23db64dc63-config\") pod \"kube-apiserver-proxy-ip-10-0-131-119.ec2.internal\" (UID: \"148fdac59207384bb5b95d23db64dc63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-119.ec2.internal" Jul 31 23:46:22.623152 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:22.623077 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:23.803659 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:23.803638 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" Jul 31 23:46:23.808571 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:23.808558 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-119.ec2.internal" Jul 31 23:46:23.809648 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:23.809631 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:23.910567 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:23.910528 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:24.011142 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:24.011063 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:24.111639 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:24.111599 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:24.166147 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.166120 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 31 23:46:24.166604 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.166261 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 31 23:46:24.166604 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.166270 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 31 23:46:24.212484 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:24.212452 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:24.224738 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.224704 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-30 23:41:23 +0000 UTC" deadline="2028-02-10 00:02:52.265733136 +0000 UTC" Jul 31 23:46:24.224738 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.224736 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13392h16m28.04100191s" Jul 31 23:46:24.313486 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:24.313454 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:24.377366 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:24.377328 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod148fdac59207384bb5b95d23db64dc63.slice/crio-d5c3ac5b966bc7219dd495c392c87ee231e44a1c60523165920de2b600d4ec90 WatchSource:0}: Error finding container d5c3ac5b966bc7219dd495c392c87ee231e44a1c60523165920de2b600d4ec90: Status 404 returned error can't find the container with id d5c3ac5b966bc7219dd495c392c87ee231e44a1c60523165920de2b600d4ec90 Jul 31 23:46:24.377810 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:24.377787 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde477c8056e974ae14ba31ef7a62870f.slice/crio-a3a6630d466d23e1cf792869bdd8b549460c6ee1e111203d6e031e98f6d5800e WatchSource:0}: Error finding container a3a6630d466d23e1cf792869bdd8b549460c6ee1e111203d6e031e98f6d5800e: Status 404 returned error can't find the container with id a3a6630d466d23e1cf792869bdd8b549460c6ee1e111203d6e031e98f6d5800e Jul 31 23:46:24.382261 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.382242 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 23:46:24.395271 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.395253 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 23:46:24.413825 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:24.413799 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:24.514388 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:24.514357 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:24.614948 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:24.614880 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-119.ec2.internal\" not found" Jul 31 23:46:24.642672 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.642649 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 23:46:24.733085 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.733057 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" Jul 31 23:46:24.744526 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.744502 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 31 23:46:24.745717 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.745697 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-119.ec2.internal" Jul 31 23:46:24.756031 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:24.756009 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 31 23:46:25.213066 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.213035 2582 apiserver.go:52] "Watching apiserver" Jul 31 23:46:25.221717 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.221696 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 31 23:46:25.223375 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.223350 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-jr47x","kube-system/kube-apiserver-proxy-ip-10-0-131-119.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw","openshift-cluster-node-tuning-operator/tuned-g6zzk","openshift-image-registry/node-ca-qwln9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal","openshift-network-operator/iptables-alerter-8mddg","kube-system/global-pull-secret-syncer-l9sh5","kube-system/konnectivity-agent-z6tfz","openshift-dns/node-resolver-6czd7","openshift-multus/multus-additional-cni-plugins-smx29","openshift-multus/multus-tmzjt","openshift-multus/network-metrics-daemon-dbgxq","openshift-network-diagnostics/network-check-target-wlkds"] Jul 31 23:46:25.224767 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.224750 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8mddg" Jul 31 23:46:25.226346 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.226312 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:25.226444 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.226376 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:25.227293 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.227274 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zqdpm\"" Jul 31 23:46:25.227676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.227478 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.227913 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.227891 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 31 23:46:25.227998 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.227952 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 31 23:46:25.227998 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.227972 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 31 23:46:25.228838 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.228790 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.229779 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.229749 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 31 23:46:25.229779 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.229759 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 31 23:46:25.230132 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.230104 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qwln9" Jul 31 23:46:25.230481 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.230462 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 31 23:46:25.230666 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.230650 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s2dww\"" Jul 31 23:46:25.231247 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.231229 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 31 23:46:25.231247 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.231235 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 31 23:46:25.231388 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.231303 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r9mhn\"" Jul 31 23:46:25.231692 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.231675 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:25.231773 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.231724 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:25.232803 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.232781 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 31 23:46:25.233325 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.232905 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:25.233325 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.233077 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 31 23:46:25.233325 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.233261 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 31 23:46:25.233507 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.233454 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 31 23:46:25.233557 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.233512 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cbzwg\"" Jul 31 23:46:25.234145 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.234128 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.235014 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.235000 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 31 23:46:25.235335 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.235315 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9v2lp\"" Jul 31 23:46:25.235485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.235470 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:25.235568 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.235526 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:25.235624 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.235576 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 31 23:46:25.236160 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.236145 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 23:46:25.236462 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.236445 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 31 23:46:25.236554 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.236504 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 31 23:46:25.236739 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.236723 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xxtcd\"" Jul 31 23:46:25.236807 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.236767 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6czd7" Jul 31 23:46:25.237873 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.237857 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 31 23:46:25.237970 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.237889 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 31 23:46:25.237970 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.237859 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 31 23:46:25.237970 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.237938 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 31 23:46:25.238362 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.238340 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.239115 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.239090 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 31 23:46:25.239198 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.239161 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wbb5j\"" Jul 31 23:46:25.239261 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.239161 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 31 23:46:25.239959 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.239940 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.240657 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.240634 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 31 23:46:25.240936 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.240913 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 31 23:46:25.241041 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.241026 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 31 23:46:25.241110 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.241038 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 31 23:46:25.241327 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.241291 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 31 23:46:25.242066 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.241968 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dczv6\"" Jul 31 23:46:25.242562 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.242543 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4g4ll\"" Jul 31 23:46:25.242769 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.242751 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 31 23:46:25.244972 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.244951 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-sys\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.245069 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.244983 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a029017-22f0-47b7-aaef-ce8d7b3777ed-tmp\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.245069 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245003 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-kubelet-config\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:25.245069 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245023 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2k5p\" (UniqueName: \"kubernetes.io/projected/2981911a-31f1-4d77-ab06-d45688555c73-kube-api-access-z2k5p\") pod \"iptables-alerter-8mddg\" (UID: \"2981911a-31f1-4d77-ab06-d45688555c73\") " pod="openshift-network-operator/iptables-alerter-8mddg" Jul 31 23:46:25.245069 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245040 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-sysconfig\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.245069 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245063 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-run\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.245258 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245077 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-kubelet\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.245258 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245093 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2981911a-31f1-4d77-ab06-d45688555c73-iptables-alerter-script\") pod \"iptables-alerter-8mddg\" (UID: \"2981911a-31f1-4d77-ab06-d45688555c73\") " pod="openshift-network-operator/iptables-alerter-8mddg" Jul 31 23:46:25.245258 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245126 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-slash\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.245258 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245157 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/80ad104b-689c-49ca-ad79-0a682d464661-ovnkube-script-lib\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.245258 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245183 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-lib-modules\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.245258 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245205 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/80ad104b-689c-49ca-ad79-0a682d464661-ovnkube-config\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.245258 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245244 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfkp9\" (UniqueName: \"kubernetes.io/projected/2262118f-b73c-4dbf-9ca6-d14c05495100-kube-api-access-kfkp9\") pod \"node-ca-qwln9\" (UID: \"2262118f-b73c-4dbf-9ca6-d14c05495100\") " pod="openshift-image-registry/node-ca-qwln9" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245261 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-var-lib-openvswitch\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245281 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-run-ovn\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245317 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-cni-bin\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245355 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-etc-selinux\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245382 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245401 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-systemd-units\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245438 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-run-openvswitch\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245462 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-node-log\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245493 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245527 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245543 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-device-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.245572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245559 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-sys-fs\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.246030 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245575 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-sysctl-conf\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.246030 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245608 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2262118f-b73c-4dbf-9ca6-d14c05495100-host\") pod \"node-ca-qwln9\" (UID: \"2262118f-b73c-4dbf-9ca6-d14c05495100\") " pod="openshift-image-registry/node-ca-qwln9" Jul 31 23:46:25.246030 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245644 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-log-socket\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.246030 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245692 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-registration-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.246030 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245718 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-modprobe-d\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.246030 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.245759 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-host\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.246318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246034 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-run-netns\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.246318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246093 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-etc-openvswitch\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.246318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246300 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-systemd\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.246469 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246347 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-tuned\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.246469 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246426 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c911e907-1c9e-4f5a-b131-5586c281a7ed-agent-certs\") pod \"konnectivity-agent-z6tfz\" (UID: \"c911e907-1c9e-4f5a-b131-5586c281a7ed\") " pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:25.246554 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246493 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mw9p\" (UniqueName: \"kubernetes.io/projected/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-kube-api-access-7mw9p\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:25.246554 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246524 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86t8n\" (UniqueName: \"kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n\") pod \"network-check-target-wlkds\" (UID: \"49904e8b-d888-442a-bab8-1e3c3f83d2e7\") " pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:25.246626 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246572 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-cni-netd\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.246674 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246648 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkljx\" (UniqueName: \"kubernetes.io/projected/80ad104b-689c-49ca-ad79-0a682d464661-kube-api-access-hkljx\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.246723 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246685 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqx7b\" (UniqueName: \"kubernetes.io/projected/cb17d370-4bb1-48ba-a05b-8e826c797cdc-kube-api-access-mqx7b\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.246770 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246718 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-var-lib-kubelet\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.246770 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246751 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c911e907-1c9e-4f5a-b131-5586c281a7ed-konnectivity-ca\") pod \"konnectivity-agent-z6tfz\" (UID: \"c911e907-1c9e-4f5a-b131-5586c281a7ed\") " pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:25.246866 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246780 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.246866 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246806 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2262118f-b73c-4dbf-9ca6-d14c05495100-serviceca\") pod \"node-ca-qwln9\" (UID: \"2262118f-b73c-4dbf-9ca6-d14c05495100\") " pod="openshift-image-registry/node-ca-qwln9" Jul 31 23:46:25.246866 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246840 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-run-ovn-kubernetes\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.247004 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246872 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/80ad104b-689c-49ca-ad79-0a682d464661-env-overrides\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.247004 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246903 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2981911a-31f1-4d77-ab06-d45688555c73-host-slash\") pod \"iptables-alerter-8mddg\" (UID: \"2981911a-31f1-4d77-ab06-d45688555c73\") " pod="openshift-network-operator/iptables-alerter-8mddg" Jul 31 23:46:25.247004 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.246949 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 31 23:46:25.247186 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.247011 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-socket-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.247186 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.247056 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-kubernetes\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.247186 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.247084 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-run-systemd\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.247323 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.247201 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/80ad104b-689c-49ca-ad79-0a682d464661-ovn-node-metrics-cert\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.247323 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.247238 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-dbus\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:25.247323 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.247268 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-sysctl-d\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.247475 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.247334 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjjnt\" (UniqueName: \"kubernetes.io/projected/4a029017-22f0-47b7-aaef-ce8d7b3777ed-kube-api-access-tjjnt\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.265867 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.265842 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kl2kd" Jul 31 23:46:25.273788 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.273769 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kl2kd" Jul 31 23:46:25.335514 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.335489 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 31 23:46:25.348423 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348382 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-slash\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.348570 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348429 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/80ad104b-689c-49ca-ad79-0a682d464661-ovnkube-script-lib\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.348570 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348458 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-969tv\" (UniqueName: \"kubernetes.io/projected/f03a7f95-a27b-408a-afad-c7f07ac5bb58-kube-api-access-969tv\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.348570 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348478 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76114bf1-387d-413b-8b61-9cd4d084d3d9-cni-binary-copy\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.348570 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348478 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-slash\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.348570 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-lib-modules\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.348570 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348523 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/80ad104b-689c-49ca-ad79-0a682d464661-ovnkube-config\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.348851 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348562 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76114bf1-387d-413b-8b61-9cd4d084d3d9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.348851 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348627 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfkp9\" (UniqueName: \"kubernetes.io/projected/2262118f-b73c-4dbf-9ca6-d14c05495100-kube-api-access-kfkp9\") pod \"node-ca-qwln9\" (UID: \"2262118f-b73c-4dbf-9ca6-d14c05495100\") " pod="openshift-image-registry/node-ca-qwln9" Jul 31 23:46:25.348851 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348652 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-var-lib-openvswitch\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.348851 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348678 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-run-ovn\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.348851 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348725 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-var-lib-openvswitch\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.348851 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348750 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-cni-bin\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.348851 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348772 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-etc-selinux\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.348851 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348820 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:25.348851 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348827 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-lib-modules\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348847 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-systemd-units\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348870 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-cni-bin\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348890 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-run-openvswitch\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348916 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-node-log\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348918 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-systemd-units\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.348949 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348953 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-node-log\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348974 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-etc-selinux\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348987 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-run-ovn\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.348994 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349022 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-run-openvswitch\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349090 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.349330 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs podName:9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:25.849068476 +0000 UTC m=+3.008917586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs") pod "network-metrics-daemon-dbgxq" (UID: "9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349360 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349395 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-var-lib-cni-bin\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349424 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/80ad104b-689c-49ca-ad79-0a682d464661-ovnkube-config\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349444 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-device-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349470 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-sys-fs\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.349489 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-sysctl-conf\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349529 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2262118f-b73c-4dbf-9ca6-d14c05495100-host\") pod \"node-ca-qwln9\" (UID: \"2262118f-b73c-4dbf-9ca6-d14c05495100\") " pod="openshift-image-registry/node-ca-qwln9" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349532 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-sys-fs\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.349548 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret podName:03fa2c21-f2f9-450a-ac02-5a0de6eef3ee nodeName:}" failed. No retries permitted until 2026-07-31 23:46:25.849526919 +0000 UTC m=+3.009376014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret") pod "global-pull-secret-syncer-l9sh5" (UID: "03fa2c21-f2f9-450a-ac02-5a0de6eef3ee") : object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-device-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349566 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2262118f-b73c-4dbf-9ca6-d14c05495100-host\") pod \"node-ca-qwln9\" (UID: \"2262118f-b73c-4dbf-9ca6-d14c05495100\") " pod="openshift-image-registry/node-ca-qwln9" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349578 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-log-socket\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349605 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-log-socket\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.349740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349604 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-cni-dir\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-daemon-config\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349654 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-sysctl-conf\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349658 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-cnibin\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349686 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-registration-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-modprobe-d\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349730 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-host\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349751 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-run-netns\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349774 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-etc-openvswitch\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349804 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-host\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349813 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-etc-openvswitch\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349820 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-run-netns\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349845 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-modprobe-d\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349827 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-registration-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349856 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-os-release\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349893 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-etc-kubernetes\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349882 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/80ad104b-689c-49ca-ad79-0a682d464661-ovnkube-script-lib\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.350318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349920 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-systemd\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349943 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-tuned\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349963 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-socket-dir-parent\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349989 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.349997 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-systemd\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350013 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c911e907-1c9e-4f5a-b131-5586c281a7ed-agent-certs\") pod \"konnectivity-agent-z6tfz\" (UID: \"c911e907-1c9e-4f5a-b131-5586c281a7ed\") " pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350040 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mw9p\" (UniqueName: \"kubernetes.io/projected/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-kube-api-access-7mw9p\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350110 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86t8n\" (UniqueName: \"kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n\") pod \"network-check-target-wlkds\" (UID: \"49904e8b-d888-442a-bab8-1e3c3f83d2e7\") " pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350134 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-cni-netd\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350159 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkljx\" (UniqueName: \"kubernetes.io/projected/80ad104b-689c-49ca-ad79-0a682d464661-kube-api-access-hkljx\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350181 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16ff1b3d-8225-4e5e-a2df-3ea9b0205971-tmp-dir\") pod \"node-resolver-6czd7\" (UID: \"16ff1b3d-8225-4e5e-a2df-3ea9b0205971\") " pod="openshift-dns/node-resolver-6czd7" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350202 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-system-cni-dir\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350247 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-var-lib-kubelet\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350253 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-cni-netd\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350282 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqx7b\" (UniqueName: \"kubernetes.io/projected/cb17d370-4bb1-48ba-a05b-8e826c797cdc-kube-api-access-mqx7b\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350277 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350309 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-var-lib-kubelet\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.350946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350333 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c911e907-1c9e-4f5a-b131-5586c281a7ed-konnectivity-ca\") pod \"konnectivity-agent-z6tfz\" (UID: \"c911e907-1c9e-4f5a-b131-5586c281a7ed\") " pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350358 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/16ff1b3d-8225-4e5e-a2df-3ea9b0205971-hosts-file\") pod \"node-resolver-6czd7\" (UID: \"16ff1b3d-8225-4e5e-a2df-3ea9b0205971\") " pod="openshift-dns/node-resolver-6czd7" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350381 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-os-release\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350421 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-run-netns\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350443 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-var-lib-kubelet\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350451 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350476 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2262118f-b73c-4dbf-9ca6-d14c05495100-serviceca\") pod \"node-ca-qwln9\" (UID: \"2262118f-b73c-4dbf-9ca6-d14c05495100\") " pod="openshift-image-registry/node-ca-qwln9" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350513 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-run-ovn-kubernetes\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350539 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/80ad104b-689c-49ca-ad79-0a682d464661-env-overrides\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350583 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350630 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2981911a-31f1-4d77-ab06-d45688555c73-host-slash\") pod \"iptables-alerter-8mddg\" (UID: \"2981911a-31f1-4d77-ab06-d45688555c73\") " pod="openshift-network-operator/iptables-alerter-8mddg" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350669 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-run-k8s-cni-cncf-io\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-run-multus-certs\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350729 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-socket-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350748 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-kubernetes\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350769 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-run-systemd\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350802 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/80ad104b-689c-49ca-ad79-0a682d464661-ovn-node-metrics-cert\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.351676 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350830 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-dbus\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350847 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2262118f-b73c-4dbf-9ca6-d14c05495100-serviceca\") pod \"node-ca-qwln9\" (UID: \"2262118f-b73c-4dbf-9ca6-d14c05495100\") " pod="openshift-image-registry/node-ca-qwln9" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350854 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-cnibin\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350875 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-var-lib-cni-multus\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350894 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-conf-dir\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350900 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-run-systemd\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350910 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c911e907-1c9e-4f5a-b131-5586c281a7ed-konnectivity-ca\") pod \"konnectivity-agent-z6tfz\" (UID: \"c911e907-1c9e-4f5a-b131-5586c281a7ed\") " pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350913 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-sysctl-d\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350971 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2981911a-31f1-4d77-ab06-d45688555c73-host-slash\") pod \"iptables-alerter-8mddg\" (UID: \"2981911a-31f1-4d77-ab06-d45688555c73\") " pod="openshift-network-operator/iptables-alerter-8mddg" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350983 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/80ad104b-689c-49ca-ad79-0a682d464661-env-overrides\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.350982 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjjnt\" (UniqueName: \"kubernetes.io/projected/4a029017-22f0-47b7-aaef-ce8d7b3777ed-kube-api-access-tjjnt\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351026 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-hostroot\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351048 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-system-cni-dir\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351078 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8w2l\" (UniqueName: \"kubernetes.io/projected/16ff1b3d-8225-4e5e-a2df-3ea9b0205971-kube-api-access-s8w2l\") pod \"node-resolver-6czd7\" (UID: \"16ff1b3d-8225-4e5e-a2df-3ea9b0205971\") " pod="openshift-dns/node-resolver-6czd7" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351103 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f03a7f95-a27b-408a-afad-c7f07ac5bb58-cni-binary-copy\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351126 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-dbus\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351127 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76114bf1-387d-413b-8b61-9cd4d084d3d9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.352512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351172 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-sys\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351192 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a029017-22f0-47b7-aaef-ce8d7b3777ed-tmp\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351226 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-kubelet-config\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351227 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb17d370-4bb1-48ba-a05b-8e826c797cdc-socket-dir\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351251 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-sysctl-d\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351275 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-kubernetes\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351307 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2k5p\" (UniqueName: \"kubernetes.io/projected/2981911a-31f1-4d77-ab06-d45688555c73-kube-api-access-z2k5p\") pod \"iptables-alerter-8mddg\" (UID: \"2981911a-31f1-4d77-ab06-d45688555c73\") " pod="openshift-network-operator/iptables-alerter-8mddg" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351317 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-kubelet-config\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351339 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch5r6\" (UniqueName: \"kubernetes.io/projected/76114bf1-387d-413b-8b61-9cd4d084d3d9-kube-api-access-ch5r6\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351374 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-sys\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351401 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-sysconfig\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351464 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-run-ovn-kubernetes\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351447 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-run\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351530 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-sysconfig\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351571 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4a029017-22f0-47b7-aaef-ce8d7b3777ed-run\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351615 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-kubelet\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351646 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2981911a-31f1-4d77-ab06-d45688555c73-iptables-alerter-script\") pod \"iptables-alerter-8mddg\" (UID: \"2981911a-31f1-4d77-ab06-d45688555c73\") " pod="openshift-network-operator/iptables-alerter-8mddg" Jul 31 23:46:25.353173 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.351712 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/80ad104b-689c-49ca-ad79-0a682d464661-host-kubelet\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.353941 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.352301 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2981911a-31f1-4d77-ab06-d45688555c73-iptables-alerter-script\") pod \"iptables-alerter-8mddg\" (UID: \"2981911a-31f1-4d77-ab06-d45688555c73\") " pod="openshift-network-operator/iptables-alerter-8mddg" Jul 31 23:46:25.354388 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.354364 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4a029017-22f0-47b7-aaef-ce8d7b3777ed-etc-tuned\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.354524 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.354397 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a029017-22f0-47b7-aaef-ce8d7b3777ed-tmp\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.354631 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.354607 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c911e907-1c9e-4f5a-b131-5586c281a7ed-agent-certs\") pod \"konnectivity-agent-z6tfz\" (UID: \"c911e907-1c9e-4f5a-b131-5586c281a7ed\") " pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:25.354987 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.354968 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/80ad104b-689c-49ca-ad79-0a682d464661-ovn-node-metrics-cert\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.356835 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.356813 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 23:46:25.356835 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.356837 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 23:46:25.356976 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.356851 2582 projected.go:196] Error preparing data for projected volume kube-api-access-86t8n for pod openshift-network-diagnostics/network-check-target-wlkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:25.356976 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.356941 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n podName:49904e8b-d888-442a-bab8-1e3c3f83d2e7 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:25.856922896 +0000 UTC m=+3.016771988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-86t8n" (UniqueName: "kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n") pod "network-check-target-wlkds" (UID: "49904e8b-d888-442a-bab8-1e3c3f83d2e7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:25.357661 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.357639 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfkp9\" (UniqueName: \"kubernetes.io/projected/2262118f-b73c-4dbf-9ca6-d14c05495100-kube-api-access-kfkp9\") pod \"node-ca-qwln9\" (UID: \"2262118f-b73c-4dbf-9ca6-d14c05495100\") " pod="openshift-image-registry/node-ca-qwln9" Jul 31 23:46:25.358808 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.358784 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjjnt\" (UniqueName: \"kubernetes.io/projected/4a029017-22f0-47b7-aaef-ce8d7b3777ed-kube-api-access-tjjnt\") pod \"tuned-g6zzk\" (UID: \"4a029017-22f0-47b7-aaef-ce8d7b3777ed\") " pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.358968 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.358929 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mw9p\" (UniqueName: \"kubernetes.io/projected/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-kube-api-access-7mw9p\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:25.359492 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.359471 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkljx\" (UniqueName: \"kubernetes.io/projected/80ad104b-689c-49ca-ad79-0a682d464661-kube-api-access-hkljx\") pod \"ovnkube-node-jr47x\" (UID: \"80ad104b-689c-49ca-ad79-0a682d464661\") " pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.359583 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.359505 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2k5p\" (UniqueName: \"kubernetes.io/projected/2981911a-31f1-4d77-ab06-d45688555c73-kube-api-access-z2k5p\") pod \"iptables-alerter-8mddg\" (UID: \"2981911a-31f1-4d77-ab06-d45688555c73\") " pod="openshift-network-operator/iptables-alerter-8mddg" Jul 31 23:46:25.360106 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.360086 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqx7b\" (UniqueName: \"kubernetes.io/projected/cb17d370-4bb1-48ba-a05b-8e826c797cdc-kube-api-access-mqx7b\") pod \"aws-ebs-csi-driver-node-psffw\" (UID: \"cb17d370-4bb1-48ba-a05b-8e826c797cdc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.370174 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.370122 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" event={"ID":"de477c8056e974ae14ba31ef7a62870f","Type":"ContainerStarted","Data":"a3a6630d466d23e1cf792869bdd8b549460c6ee1e111203d6e031e98f6d5800e"} Jul 31 23:46:25.371208 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.371181 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-119.ec2.internal" event={"ID":"148fdac59207384bb5b95d23db64dc63","Type":"ContainerStarted","Data":"d5c3ac5b966bc7219dd495c392c87ee231e44a1c60523165920de2b600d4ec90"} Jul 31 23:46:25.385544 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.385519 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 23:46:25.452879 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.452847 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-cnibin\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.452879 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.452882 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-os-release\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.452900 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-etc-kubernetes\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.452916 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-socket-dir-parent\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.452937 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.452976 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16ff1b3d-8225-4e5e-a2df-3ea9b0205971-tmp-dir\") pod \"node-resolver-6czd7\" (UID: \"16ff1b3d-8225-4e5e-a2df-3ea9b0205971\") " pod="openshift-dns/node-resolver-6czd7" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.452978 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-cnibin\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.452998 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-system-cni-dir\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453015 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-os-release\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453022 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-var-lib-kubelet\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453059 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-var-lib-kubelet\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453059 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-socket-dir-parent\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453070 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/16ff1b3d-8225-4e5e-a2df-3ea9b0205971-hosts-file\") pod \"node-resolver-6czd7\" (UID: \"16ff1b3d-8225-4e5e-a2df-3ea9b0205971\") " pod="openshift-dns/node-resolver-6czd7" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.452996 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-etc-kubernetes\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453098 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-os-release\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453120 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-run-netns\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453135 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/16ff1b3d-8225-4e5e-a2df-3ea9b0205971-hosts-file\") pod \"node-resolver-6czd7\" (UID: \"16ff1b3d-8225-4e5e-a2df-3ea9b0205971\") " pod="openshift-dns/node-resolver-6czd7" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453150 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-run-k8s-cni-cncf-io\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453169 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-run-netns\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453174 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-run-multus-certs\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453175 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-system-cni-dir\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453203 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-cnibin\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453227 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-run-multus-certs\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453228 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-var-lib-cni-multus\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453259 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-var-lib-cni-multus\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453226 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-os-release\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453205 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-run-k8s-cni-cncf-io\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453277 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-cnibin\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453296 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-conf-dir\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453260 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-conf-dir\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453325 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-hostroot\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453335 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16ff1b3d-8225-4e5e-a2df-3ea9b0205971-tmp-dir\") pod \"node-resolver-6czd7\" (UID: \"16ff1b3d-8225-4e5e-a2df-3ea9b0205971\") " pod="openshift-dns/node-resolver-6czd7" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453340 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-system-cni-dir\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.453858 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453363 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-system-cni-dir\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453381 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-hostroot\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453382 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8w2l\" (UniqueName: \"kubernetes.io/projected/16ff1b3d-8225-4e5e-a2df-3ea9b0205971-kube-api-access-s8w2l\") pod \"node-resolver-6czd7\" (UID: \"16ff1b3d-8225-4e5e-a2df-3ea9b0205971\") " pod="openshift-dns/node-resolver-6czd7" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453432 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f03a7f95-a27b-408a-afad-c7f07ac5bb58-cni-binary-copy\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453456 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76114bf1-387d-413b-8b61-9cd4d084d3d9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453485 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch5r6\" (UniqueName: \"kubernetes.io/projected/76114bf1-387d-413b-8b61-9cd4d084d3d9-kube-api-access-ch5r6\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453551 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-969tv\" (UniqueName: \"kubernetes.io/projected/f03a7f95-a27b-408a-afad-c7f07ac5bb58-kube-api-access-969tv\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453583 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76114bf1-387d-413b-8b61-9cd4d084d3d9-cni-binary-copy\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453659 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76114bf1-387d-413b-8b61-9cd4d084d3d9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453691 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76114bf1-387d-413b-8b61-9cd4d084d3d9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453776 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-var-lib-cni-bin\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453805 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-cni-dir\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453830 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-daemon-config\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.453963 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-host-var-lib-cni-bin\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.454023 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-cni-dir\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.454091 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f03a7f95-a27b-408a-afad-c7f07ac5bb58-cni-binary-copy\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.454112 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76114bf1-387d-413b-8b61-9cd4d084d3d9-cni-binary-copy\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.454710 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.454157 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76114bf1-387d-413b-8b61-9cd4d084d3d9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.455562 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.454193 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76114bf1-387d-413b-8b61-9cd4d084d3d9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.455562 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.454304 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f03a7f95-a27b-408a-afad-c7f07ac5bb58-multus-daemon-config\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.462138 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.462119 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch5r6\" (UniqueName: \"kubernetes.io/projected/76114bf1-387d-413b-8b61-9cd4d084d3d9-kube-api-access-ch5r6\") pod \"multus-additional-cni-plugins-smx29\" (UID: \"76114bf1-387d-413b-8b61-9cd4d084d3d9\") " pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.462317 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.462299 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-969tv\" (UniqueName: \"kubernetes.io/projected/f03a7f95-a27b-408a-afad-c7f07ac5bb58-kube-api-access-969tv\") pod \"multus-tmzjt\" (UID: \"f03a7f95-a27b-408a-afad-c7f07ac5bb58\") " pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.462390 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.462346 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8w2l\" (UniqueName: \"kubernetes.io/projected/16ff1b3d-8225-4e5e-a2df-3ea9b0205971-kube-api-access-s8w2l\") pod \"node-resolver-6czd7\" (UID: \"16ff1b3d-8225-4e5e-a2df-3ea9b0205971\") " pod="openshift-dns/node-resolver-6czd7" Jul 31 23:46:25.538492 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.538398 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8mddg" Jul 31 23:46:25.545476 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.545454 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" Jul 31 23:46:25.553399 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.553375 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" Jul 31 23:46:25.559269 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.559249 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qwln9" Jul 31 23:46:25.565970 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.565946 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:25.572445 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.572427 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:25.579204 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.579180 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6czd7" Jul 31 23:46:25.586665 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.586645 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-smx29" Jul 31 23:46:25.591212 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.591195 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tmzjt" Jul 31 23:46:25.857893 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.857575 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86t8n\" (UniqueName: \"kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n\") pod \"network-check-target-wlkds\" (UID: \"49904e8b-d888-442a-bab8-1e3c3f83d2e7\") " pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:25.857893 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.857652 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:25.857893 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:25.857684 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:25.857893 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.857802 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:25.858171 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.857896 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret podName:03fa2c21-f2f9-450a-ac02-5a0de6eef3ee nodeName:}" failed. No retries permitted until 2026-07-31 23:46:26.857866336 +0000 UTC m=+4.017715442 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret") pod "global-pull-secret-syncer-l9sh5" (UID: "03fa2c21-f2f9-450a-ac02-5a0de6eef3ee") : object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:25.858171 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.857971 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:25.858171 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.858028 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs podName:9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:26.85801323 +0000 UTC m=+4.017862320 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs") pod "network-metrics-daemon-dbgxq" (UID: "9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:25.858171 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.858044 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 23:46:25.858171 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.858061 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 23:46:25.858171 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.858070 2582 projected.go:196] Error preparing data for projected volume kube-api-access-86t8n for pod openshift-network-diagnostics/network-check-target-wlkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:25.858171 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:25.858118 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n podName:49904e8b-d888-442a-bab8-1e3c3f83d2e7 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:26.858109189 +0000 UTC m=+4.017958279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-86t8n" (UniqueName: "kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n") pod "network-check-target-wlkds" (UID: "49904e8b-d888-442a-bab8-1e3c3f83d2e7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:26.173928 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:26.173852 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf03a7f95_a27b_408a_afad_c7f07ac5bb58.slice/crio-52086c15f12e0352ec7fd930427bf805eb3470538a2a3a8995d94dd611715200 WatchSource:0}: Error finding container 52086c15f12e0352ec7fd930427bf805eb3470538a2a3a8995d94dd611715200: Status 404 returned error can't find the container with id 52086c15f12e0352ec7fd930427bf805eb3470538a2a3a8995d94dd611715200 Jul 31 23:46:26.175473 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:26.175447 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2981911a_31f1_4d77_ab06_d45688555c73.slice/crio-75fefe326674503526e947b93740a90c1e7832d4a5da9dbf3df5f5f0b6c9b43d WatchSource:0}: Error finding container 75fefe326674503526e947b93740a90c1e7832d4a5da9dbf3df5f5f0b6c9b43d: Status 404 returned error can't find the container with id 75fefe326674503526e947b93740a90c1e7832d4a5da9dbf3df5f5f0b6c9b43d Jul 31 23:46:26.176291 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:26.176264 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc911e907_1c9e_4f5a_b131_5586c281a7ed.slice/crio-917c1b0068d06c02cb3bc0b6426f246228cb580fcca48e6958513a7c3c493030 WatchSource:0}: Error finding container 917c1b0068d06c02cb3bc0b6426f246228cb580fcca48e6958513a7c3c493030: Status 404 returned error can't find the container with id 917c1b0068d06c02cb3bc0b6426f246228cb580fcca48e6958513a7c3c493030 Jul 31 23:46:26.177740 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:26.177627 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ff1b3d_8225_4e5e_a2df_3ea9b0205971.slice/crio-9a2c8b84d58fd692bc05f781ce5b7a487337557b90b730e115b49e15565ad441 WatchSource:0}: Error finding container 9a2c8b84d58fd692bc05f781ce5b7a487337557b90b730e115b49e15565ad441: Status 404 returned error can't find the container with id 9a2c8b84d58fd692bc05f781ce5b7a487337557b90b730e115b49e15565ad441 Jul 31 23:46:26.178473 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:26.178448 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80ad104b_689c_49ca_ad79_0a682d464661.slice/crio-3b9727f52c02c36f4ac61322a183dfc3b2154a9cff4357fc0d1746acb0d46471 WatchSource:0}: Error finding container 3b9727f52c02c36f4ac61322a183dfc3b2154a9cff4357fc0d1746acb0d46471: Status 404 returned error can't find the container with id 3b9727f52c02c36f4ac61322a183dfc3b2154a9cff4357fc0d1746acb0d46471 Jul 31 23:46:26.181287 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:26.181257 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2262118f_b73c_4dbf_9ca6_d14c05495100.slice/crio-9c22d2cbb6a2fe692519d5f71529e9963f1dd830ca857f1559978bc26b12f699 WatchSource:0}: Error finding container 9c22d2cbb6a2fe692519d5f71529e9963f1dd830ca857f1559978bc26b12f699: Status 404 returned error can't find the container with id 9c22d2cbb6a2fe692519d5f71529e9963f1dd830ca857f1559978bc26b12f699 Jul 31 23:46:26.183749 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:26.183728 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a029017_22f0_47b7_aaef_ce8d7b3777ed.slice/crio-eca7a25c7de01147df6e76f0df7b8c3763c886925d257f7901520ed7c0af015e WatchSource:0}: Error finding container eca7a25c7de01147df6e76f0df7b8c3763c886925d257f7901520ed7c0af015e: Status 404 returned error can't find the container with id eca7a25c7de01147df6e76f0df7b8c3763c886925d257f7901520ed7c0af015e Jul 31 23:46:26.184869 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:26.184853 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76114bf1_387d_413b_8b61_9cd4d084d3d9.slice/crio-d963c9b4b21301b44f0854cca055d636fd80e91258fe9e5dd98c462d892140d3 WatchSource:0}: Error finding container d963c9b4b21301b44f0854cca055d636fd80e91258fe9e5dd98c462d892140d3: Status 404 returned error can't find the container with id d963c9b4b21301b44f0854cca055d636fd80e91258fe9e5dd98c462d892140d3 Jul 31 23:46:26.186300 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:26.186278 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb17d370_4bb1_48ba_a05b_8e826c797cdc.slice/crio-c91e3332111faef453b2406bc62f95a2c5e7cf4bda162d1e01b0fe1b94f16678 WatchSource:0}: Error finding container c91e3332111faef453b2406bc62f95a2c5e7cf4bda162d1e01b0fe1b94f16678: Status 404 returned error can't find the container with id c91e3332111faef453b2406bc62f95a2c5e7cf4bda162d1e01b0fe1b94f16678 Jul 31 23:46:26.276326 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.276165 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 23:41:25 +0000 UTC" deadline="2028-02-12 16:42:42.79042483 +0000 UTC" Jul 31 23:46:26.276326 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.276323 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13456h56m16.514105203s" Jul 31 23:46:26.365846 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.365818 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:26.366000 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:26.365975 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:26.373912 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.373876 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" event={"ID":"cb17d370-4bb1-48ba-a05b-8e826c797cdc","Type":"ContainerStarted","Data":"c91e3332111faef453b2406bc62f95a2c5e7cf4bda162d1e01b0fe1b94f16678"} Jul 31 23:46:26.374949 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.374923 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" event={"ID":"4a029017-22f0-47b7-aaef-ce8d7b3777ed","Type":"ContainerStarted","Data":"eca7a25c7de01147df6e76f0df7b8c3763c886925d257f7901520ed7c0af015e"} Jul 31 23:46:26.375954 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.375934 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qwln9" event={"ID":"2262118f-b73c-4dbf-9ca6-d14c05495100","Type":"ContainerStarted","Data":"9c22d2cbb6a2fe692519d5f71529e9963f1dd830ca857f1559978bc26b12f699"} Jul 31 23:46:26.376939 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.376913 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8mddg" event={"ID":"2981911a-31f1-4d77-ab06-d45688555c73","Type":"ContainerStarted","Data":"75fefe326674503526e947b93740a90c1e7832d4a5da9dbf3df5f5f0b6c9b43d"} Jul 31 23:46:26.378016 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.377988 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" event={"ID":"80ad104b-689c-49ca-ad79-0a682d464661","Type":"ContainerStarted","Data":"3b9727f52c02c36f4ac61322a183dfc3b2154a9cff4357fc0d1746acb0d46471"} Jul 31 23:46:26.378889 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.378869 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6czd7" event={"ID":"16ff1b3d-8225-4e5e-a2df-3ea9b0205971","Type":"ContainerStarted","Data":"9a2c8b84d58fd692bc05f781ce5b7a487337557b90b730e115b49e15565ad441"} Jul 31 23:46:26.379976 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.379955 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smx29" event={"ID":"76114bf1-387d-413b-8b61-9cd4d084d3d9","Type":"ContainerStarted","Data":"d963c9b4b21301b44f0854cca055d636fd80e91258fe9e5dd98c462d892140d3"} Jul 31 23:46:26.380967 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.380944 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z6tfz" event={"ID":"c911e907-1c9e-4f5a-b131-5586c281a7ed","Type":"ContainerStarted","Data":"917c1b0068d06c02cb3bc0b6426f246228cb580fcca48e6958513a7c3c493030"} Jul 31 23:46:26.381955 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.381932 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tmzjt" event={"ID":"f03a7f95-a27b-408a-afad-c7f07ac5bb58","Type":"ContainerStarted","Data":"52086c15f12e0352ec7fd930427bf805eb3470538a2a3a8995d94dd611715200"} Jul 31 23:46:26.383634 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.383615 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-119.ec2.internal" event={"ID":"148fdac59207384bb5b95d23db64dc63","Type":"ContainerStarted","Data":"6cc88f08796e866d8cf1a48cddacc74357e44c08760d009d49b1a8396ad135bf"} Jul 31 23:46:26.398869 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.398820 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-119.ec2.internal" podStartSLOduration=2.398805748 podStartE2EDuration="2.398805748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:26.398672631 +0000 UTC m=+3.558521743" watchObservedRunningTime="2026-07-31 23:46:26.398805748 +0000 UTC m=+3.558654860" Jul 31 23:46:26.867018 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.866027 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86t8n\" (UniqueName: \"kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n\") pod \"network-check-target-wlkds\" (UID: \"49904e8b-d888-442a-bab8-1e3c3f83d2e7\") " pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:26.867018 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.866106 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:26.867018 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:26.866137 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:26.867018 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:26.866253 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:26.867018 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:26.866309 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret podName:03fa2c21-f2f9-450a-ac02-5a0de6eef3ee nodeName:}" failed. No retries permitted until 2026-07-31 23:46:28.866291026 +0000 UTC m=+6.026140135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret") pod "global-pull-secret-syncer-l9sh5" (UID: "03fa2c21-f2f9-450a-ac02-5a0de6eef3ee") : object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:26.867018 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:26.866778 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 23:46:26.867018 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:26.866796 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 23:46:26.867018 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:26.866809 2582 projected.go:196] Error preparing data for projected volume kube-api-access-86t8n for pod openshift-network-diagnostics/network-check-target-wlkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:26.867018 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:26.866852 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n podName:49904e8b-d888-442a-bab8-1e3c3f83d2e7 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:28.866837212 +0000 UTC m=+6.026686306 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-86t8n" (UniqueName: "kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n") pod "network-check-target-wlkds" (UID: "49904e8b-d888-442a-bab8-1e3c3f83d2e7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:26.867018 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:26.866912 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:26.867018 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:26.866943 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs podName:9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:28.866931434 +0000 UTC m=+6.026780529 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs") pod "network-metrics-daemon-dbgxq" (UID: "9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:27.276598 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:27.276488 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 23:41:25 +0000 UTC" deadline="2028-05-02 01:07:02.453057706 +0000 UTC" Jul 31 23:46:27.276598 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:27.276545 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15361h20m35.176517558s" Jul 31 23:46:27.369374 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:27.369343 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:27.369598 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:27.369488 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:27.369906 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:27.369863 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:27.369980 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:27.369957 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:27.391636 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:27.390479 2582 generic.go:356] "Generic (PLEG): container finished" podID="de477c8056e974ae14ba31ef7a62870f" containerID="25ad5be92c0ca213aba9662f54ca879076f6074b23e593555f61a70369ee5a3e" exitCode=0 Jul 31 23:46:27.391636 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:27.391394 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" event={"ID":"de477c8056e974ae14ba31ef7a62870f","Type":"ContainerDied","Data":"25ad5be92c0ca213aba9662f54ca879076f6074b23e593555f61a70369ee5a3e"} Jul 31 23:46:28.367007 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:28.365805 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:28.367007 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:28.365941 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:28.400432 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:28.399878 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" event={"ID":"de477c8056e974ae14ba31ef7a62870f","Type":"ContainerStarted","Data":"481428255ec7c142b9bc8a38b4b35f35266e3f70daee64e9767cd1a9e094cf3e"} Jul 31 23:46:28.418888 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:28.418829 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-119.ec2.internal" podStartSLOduration=4.418808289 podStartE2EDuration="4.418808289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:28.418538865 +0000 UTC m=+5.578387978" watchObservedRunningTime="2026-07-31 23:46:28.418808289 +0000 UTC m=+5.578657403" Jul 31 23:46:28.882257 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:28.882220 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:28.882459 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:28.882273 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:28.882459 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:28.882319 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86t8n\" (UniqueName: \"kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n\") pod \"network-check-target-wlkds\" (UID: \"49904e8b-d888-442a-bab8-1e3c3f83d2e7\") " pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:28.882459 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:28.882396 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:28.882623 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:28.882471 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 23:46:28.882623 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:28.882482 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs podName:9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:32.88246039 +0000 UTC m=+10.042309495 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs") pod "network-metrics-daemon-dbgxq" (UID: "9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:28.882623 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:28.882488 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 23:46:28.882623 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:28.882508 2582 projected.go:196] Error preparing data for projected volume kube-api-access-86t8n for pod openshift-network-diagnostics/network-check-target-wlkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:28.882623 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:28.882544 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n podName:49904e8b-d888-442a-bab8-1e3c3f83d2e7 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:32.882532044 +0000 UTC m=+10.042381137 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-86t8n" (UniqueName: "kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n") pod "network-check-target-wlkds" (UID: "49904e8b-d888-442a-bab8-1e3c3f83d2e7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:28.882623 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:28.882545 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:28.882623 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:28.882597 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret podName:03fa2c21-f2f9-450a-ac02-5a0de6eef3ee nodeName:}" failed. No retries permitted until 2026-07-31 23:46:32.88257908 +0000 UTC m=+10.042428192 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret") pod "global-pull-secret-syncer-l9sh5" (UID: "03fa2c21-f2f9-450a-ac02-5a0de6eef3ee") : object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:29.368018 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:29.367943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:29.368490 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:29.368071 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:29.368490 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:29.368200 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:29.368490 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:29.368281 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:30.366890 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:30.366436 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:30.366890 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:30.366588 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:31.366288 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:31.366197 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:31.366780 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:31.366322 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:31.366780 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:31.366207 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:31.366903 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:31.366835 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:32.366646 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:32.366601 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:32.367128 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:32.366756 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:32.916479 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:32.915557 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:32.916479 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:32.915609 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:32.916479 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:32.915647 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86t8n\" (UniqueName: \"kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n\") pod \"network-check-target-wlkds\" (UID: \"49904e8b-d888-442a-bab8-1e3c3f83d2e7\") " pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:32.916479 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:32.915792 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 23:46:32.916479 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:32.915818 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 23:46:32.916479 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:32.915830 2582 projected.go:196] Error preparing data for projected volume kube-api-access-86t8n for pod openshift-network-diagnostics/network-check-target-wlkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:32.916479 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:32.915891 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n podName:49904e8b-d888-442a-bab8-1e3c3f83d2e7 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:40.915870837 +0000 UTC m=+18.075719930 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-86t8n" (UniqueName: "kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n") pod "network-check-target-wlkds" (UID: "49904e8b-d888-442a-bab8-1e3c3f83d2e7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:32.916479 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:32.916294 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:32.916479 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:32.916344 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs podName:9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:40.916328638 +0000 UTC m=+18.076177732 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs") pod "network-metrics-daemon-dbgxq" (UID: "9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:32.916479 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:32.916396 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:32.916479 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:32.916446 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret podName:03fa2c21-f2f9-450a-ac02-5a0de6eef3ee nodeName:}" failed. No retries permitted until 2026-07-31 23:46:40.916433483 +0000 UTC m=+18.076282579 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret") pod "global-pull-secret-syncer-l9sh5" (UID: "03fa2c21-f2f9-450a-ac02-5a0de6eef3ee") : object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:33.367794 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:33.367245 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:33.367794 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:33.367352 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:33.368427 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:33.368359 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:33.368612 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:33.368574 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:34.365909 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:34.365828 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:34.366113 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:34.365971 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:35.366184 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:35.366016 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:35.366184 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:35.366016 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:35.366184 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:35.366170 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:35.366705 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:35.366220 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:36.366213 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:36.366193 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:36.366483 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:36.366297 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:37.365735 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:37.365696 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:37.365929 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:37.365829 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:37.365929 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:37.365864 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:37.366032 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:37.365979 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:38.366044 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:38.366004 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:38.366519 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:38.366126 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:39.366605 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:39.366571 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:39.367024 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:39.366697 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:39.367024 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:39.366749 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:39.367024 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:39.366821 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:40.365888 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:40.365848 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:40.366083 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:40.365964 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:40.975031 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:40.974989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:40.975539 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:40.975049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:40.975539 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:40.975094 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86t8n\" (UniqueName: \"kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n\") pod \"network-check-target-wlkds\" (UID: \"49904e8b-d888-442a-bab8-1e3c3f83d2e7\") " pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:40.975539 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:40.975164 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:40.975539 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:40.975239 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:40.975539 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:40.975252 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs podName:9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:56.97522717 +0000 UTC m=+34.135076261 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs") pod "network-metrics-daemon-dbgxq" (UID: "9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:40.975539 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:40.975258 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 23:46:40.975539 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:40.975278 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 23:46:40.975539 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:40.975292 2582 projected.go:196] Error preparing data for projected volume kube-api-access-86t8n for pod openshift-network-diagnostics/network-check-target-wlkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:40.975539 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:40.975293 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret podName:03fa2c21-f2f9-450a-ac02-5a0de6eef3ee nodeName:}" failed. No retries permitted until 2026-07-31 23:46:56.975279858 +0000 UTC m=+34.135128952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret") pod "global-pull-secret-syncer-l9sh5" (UID: "03fa2c21-f2f9-450a-ac02-5a0de6eef3ee") : object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:40.975539 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:40.975353 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n podName:49904e8b-d888-442a-bab8-1e3c3f83d2e7 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:56.975337582 +0000 UTC m=+34.135186677 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-86t8n" (UniqueName: "kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n") pod "network-check-target-wlkds" (UID: "49904e8b-d888-442a-bab8-1e3c3f83d2e7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:41.366813 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:41.366734 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:41.366813 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:41.366734 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:41.367033 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:41.366863 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:41.367033 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:41.366881 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:42.366026 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:42.365994 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:42.366479 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:42.366117 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:43.366517 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:43.366470 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:43.366957 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:43.366587 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:43.366957 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:43.366641 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:43.367514 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:43.367479 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:44.366628 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:44.366596 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:44.367023 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:44.366698 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:45.369054 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.368825 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:45.369821 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.368825 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:45.369821 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:45.369115 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:45.369821 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:45.369172 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:45.432937 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.432897 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" event={"ID":"cb17d370-4bb1-48ba-a05b-8e826c797cdc","Type":"ContainerStarted","Data":"80bd6948ac39e6e21d2f97b2816cdaeb03c238f13a7964e27d0fee4678dbc140"} Jul 31 23:46:45.434709 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.434535 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" event={"ID":"4a029017-22f0-47b7-aaef-ce8d7b3777ed","Type":"ContainerStarted","Data":"e0760cc0f2a7625b6272f9a76f4ffdff89632f225481681c84c31209fbdfc2c6"} Jul 31 23:46:45.436242 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.436202 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qwln9" event={"ID":"2262118f-b73c-4dbf-9ca6-d14c05495100","Type":"ContainerStarted","Data":"8b17182ab7dc8a358d90289565a261fa0dd113bf8e78a0808b8b10ea0a1d5075"} Jul 31 23:46:45.439598 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.439577 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" event={"ID":"80ad104b-689c-49ca-ad79-0a682d464661","Type":"ContainerStarted","Data":"0056deebaae75c0e580d946f685262ec3f1095118ccb75962e51d03f9b965fc7"} Jul 31 23:46:45.439712 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.439646 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" event={"ID":"80ad104b-689c-49ca-ad79-0a682d464661","Type":"ContainerStarted","Data":"52535447b5e4751dfe66f8b62466dd68818572b6b02ef51567e4e9755557b48f"} Jul 31 23:46:45.439712 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.439697 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" event={"ID":"80ad104b-689c-49ca-ad79-0a682d464661","Type":"ContainerStarted","Data":"2e5bfdfb695088b940125429e8dd67e15cab15d7448372fa4c32c7f3a1be5175"} Jul 31 23:46:45.439804 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.439744 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" event={"ID":"80ad104b-689c-49ca-ad79-0a682d464661","Type":"ContainerStarted","Data":"d52244690d7e726fb139c87ba70510bf331207a0364534977140ade53997ffa3"} Jul 31 23:46:45.439804 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.439793 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" event={"ID":"80ad104b-689c-49ca-ad79-0a682d464661","Type":"ContainerStarted","Data":"1182e4c9772f8017a815e41f3092c62454bbacbe254a4d9f0f9900c9dc29e6d3"} Jul 31 23:46:45.439885 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.439840 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" event={"ID":"80ad104b-689c-49ca-ad79-0a682d464661","Type":"ContainerStarted","Data":"f200d94f05039798268f5a7e95ad69e1bbc4fb85c011dcee9e3b4df1e6ed0855"} Jul 31 23:46:45.441120 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.441095 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6czd7" event={"ID":"16ff1b3d-8225-4e5e-a2df-3ea9b0205971","Type":"ContainerStarted","Data":"13c4e4a8656e894ff3122ed70df76f16a014cb0c013c6c262db20807c1f2ca71"} Jul 31 23:46:45.442738 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.442710 2582 generic.go:356] "Generic (PLEG): container finished" podID="76114bf1-387d-413b-8b61-9cd4d084d3d9" containerID="1ca8e8c417b303674c5275b1ffaff007640e274185e762ea94fdba4c90b552c9" exitCode=0 Jul 31 23:46:45.442836 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.442791 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smx29" event={"ID":"76114bf1-387d-413b-8b61-9cd4d084d3d9","Type":"ContainerDied","Data":"1ca8e8c417b303674c5275b1ffaff007640e274185e762ea94fdba4c90b552c9"} Jul 31 23:46:45.444439 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.444416 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z6tfz" event={"ID":"c911e907-1c9e-4f5a-b131-5586c281a7ed","Type":"ContainerStarted","Data":"31d21ec5ccb7ef60a46ceeddaae9f247c2535368d8d6f8ac5483cf141e5ac2c8"} Jul 31 23:46:45.446147 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.446113 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tmzjt" event={"ID":"f03a7f95-a27b-408a-afad-c7f07ac5bb58","Type":"ContainerStarted","Data":"0b6a656bc278379c58d0a3daa5ab34dbd426de0641d30a039a58acab13702a7c"} Jul 31 23:46:45.451024 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.450982 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-g6zzk" podStartSLOduration=4.204520349 podStartE2EDuration="22.450967172s" totalImagesPullingTime="18.246446823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:45.450787734 +0000 UTC m=+22.610636847" watchObservedRunningTime="2026-07-31 23:46:45.450967172 +0000 UTC m=+22.610816284" Jul 31 23:46:45.499262 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.499219 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qwln9" podStartSLOduration=4.251924302 podStartE2EDuration="22.499200827s" totalImagesPullingTime="18.247276525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:45.499112978 +0000 UTC m=+22.658962090" watchObservedRunningTime="2026-07-31 23:46:45.499200827 +0000 UTC m=+22.659049942" Jul 31 23:46:45.499520 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.499498 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tmzjt" podStartSLOduration=4.080913471 podStartE2EDuration="22.499491286s" totalImagesPullingTime="18.418577815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:45.485893235 +0000 UTC m=+22.645742346" watchObservedRunningTime="2026-07-31 23:46:45.499491286 +0000 UTC m=+22.659340397" Jul 31 23:46:45.514981 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.514945 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6czd7" podStartSLOduration=4.263775822 podStartE2EDuration="22.514931856s" totalImagesPullingTime="18.251156034s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:45.514860266 +0000 UTC m=+22.674709379" watchObservedRunningTime="2026-07-31 23:46:45.514931856 +0000 UTC m=+22.674780966" Jul 31 23:46:45.529770 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.529733 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z6tfz" podStartSLOduration=12.277924673 podStartE2EDuration="22.529720603s" totalImagesPullingTime="10.25179593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:45.529306053 +0000 UTC m=+22.689155166" watchObservedRunningTime="2026-07-31 23:46:45.529720603 +0000 UTC m=+22.689569714" Jul 31 23:46:45.915472 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:45.915399 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 31 23:46:46.297019 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:46.296816 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-31T23:46:45.915468256Z","UUID":"5fbb696e-33dd-445a-84dc-a5ec978a619a","Handler":null,"Name":"","Endpoint":""} Jul 31 23:46:46.298515 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:46.298490 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 31 23:46:46.298682 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:46.298532 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 31 23:46:46.366286 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:46.366252 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:46.366478 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:46.366368 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:46.451757 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:46.451719 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" event={"ID":"cb17d370-4bb1-48ba-a05b-8e826c797cdc","Type":"ContainerStarted","Data":"b9403df38c7b257bd8e1b00a3ca57e480008e8c8afe21f003aca6481c93d43d5"} Jul 31 23:46:46.453420 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:46.453360 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8mddg" event={"ID":"2981911a-31f1-4d77-ab06-d45688555c73","Type":"ContainerStarted","Data":"fb43b38d884a02486c338a093a1ae6183ceacf42b94280cfb95b89e883ecb096"} Jul 31 23:46:46.472134 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:46.472084 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8mddg" podStartSLOduration=5.219295735 podStartE2EDuration="23.472070325s" totalImagesPullingTime="18.25277459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:46.471539691 +0000 UTC m=+23.631388816" watchObservedRunningTime="2026-07-31 23:46:46.472070325 +0000 UTC m=+23.631919436" Jul 31 23:46:47.366632 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:47.366599 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:47.366817 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:47.366733 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:47.367237 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:47.367204 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:47.367377 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:47.367304 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:47.459073 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:47.458852 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" event={"ID":"cb17d370-4bb1-48ba-a05b-8e826c797cdc","Type":"ContainerStarted","Data":"ef5d2c0ff6f6ec5dbb3244463832d434b53c44a5bd70d458c5870b13b4632b86"} Jul 31 23:46:47.462973 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:47.462947 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" event={"ID":"80ad104b-689c-49ca-ad79-0a682d464661","Type":"ContainerStarted","Data":"c857af2905fc267b253fd25323b4b1eee0a4e23c2392199a1613b2f25d12acd2"} Jul 31 23:46:47.475425 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:47.475366 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-psffw" podStartSLOduration=4.025697923 podStartE2EDuration="24.47534813s" totalImagesPullingTime="20.449650207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:47.474767937 +0000 UTC m=+24.634617050" watchObservedRunningTime="2026-07-31 23:46:47.47534813 +0000 UTC m=+24.635197246" Jul 31 23:46:48.366202 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:48.366008 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:48.366369 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:48.366304 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:49.366498 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:49.366465 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:49.366855 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:49.366505 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:49.366855 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:49.366599 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:49.366855 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:49.366709 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:49.471052 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:49.471022 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" event={"ID":"80ad104b-689c-49ca-ad79-0a682d464661","Type":"ContainerStarted","Data":"518bf24fcb1af090bd8e134b32b9a0dbce096fb1b72c307195c0d7615ebaa4ab"} Jul 31 23:46:49.836370 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:49.836196 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:49.836727 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:49.836709 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:50.366002 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:50.365975 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:50.366187 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:50.366086 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:50.475113 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:50.475085 2582 generic.go:356] "Generic (PLEG): container finished" podID="76114bf1-387d-413b-8b61-9cd4d084d3d9" containerID="6582faa4cf3bdf599f077a87c3f99a19f7073f4fad3184fc60116dcf2f36d828" exitCode=0 Jul 31 23:46:50.476028 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:50.475176 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smx29" event={"ID":"76114bf1-387d-413b-8b61-9cd4d084d3d9","Type":"ContainerDied","Data":"6582faa4cf3bdf599f077a87c3f99a19f7073f4fad3184fc60116dcf2f36d828"} Jul 31 23:46:50.476028 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:50.475571 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:50.476028 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:50.475877 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z6tfz" Jul 31 23:46:50.539362 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:50.539313 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" podStartSLOduration=9.263731806 podStartE2EDuration="27.539298156s" totalImagesPullingTime="18.27556635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:50.539015499 +0000 UTC m=+27.698864600" watchObservedRunningTime="2026-07-31 23:46:50.539298156 +0000 UTC m=+27.699147269" Jul 31 23:46:51.366012 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:51.365988 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:51.366176 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:51.365988 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:51.366176 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:51.366089 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:51.366262 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:51.366184 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:51.397097 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:51.397070 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wlkds"] Jul 31 23:46:51.399859 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:51.399840 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dbgxq"] Jul 31 23:46:51.399970 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:51.399962 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:51.400105 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:51.400084 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:51.412146 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:51.412091 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l9sh5"] Jul 31 23:46:51.479677 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:51.479636 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smx29" event={"ID":"76114bf1-387d-413b-8b61-9cd4d084d3d9","Type":"ContainerStarted","Data":"fdabce59a63540b2cbf39d778ef05cff7d41046a24efd7317039586b6be3a2ac"} Jul 31 23:46:51.480042 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:51.479684 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:51.480042 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:51.479661 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:51.480149 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:51.480093 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:51.480199 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:51.480182 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:52.483930 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:52.483897 2582 generic.go:356] "Generic (PLEG): container finished" podID="76114bf1-387d-413b-8b61-9cd4d084d3d9" containerID="fdabce59a63540b2cbf39d778ef05cff7d41046a24efd7317039586b6be3a2ac" exitCode=0 Jul 31 23:46:52.484302 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:52.483987 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smx29" event={"ID":"76114bf1-387d-413b-8b61-9cd4d084d3d9","Type":"ContainerDied","Data":"fdabce59a63540b2cbf39d778ef05cff7d41046a24efd7317039586b6be3a2ac"} Jul 31 23:46:53.366920 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:53.366744 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:53.367079 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:53.366818 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:53.367079 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:53.367012 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:53.367079 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:53.366847 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:53.367196 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:53.367074 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:53.367237 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:53.367207 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:54.490318 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:54.490290 2582 generic.go:356] "Generic (PLEG): container finished" podID="76114bf1-387d-413b-8b61-9cd4d084d3d9" containerID="a54c93ee1391f792d4dd1bd959360862d369182ab54f972c15d27052f9adc897" exitCode=0 Jul 31 23:46:54.490987 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:54.490347 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smx29" event={"ID":"76114bf1-387d-413b-8b61-9cd4d084d3d9","Type":"ContainerDied","Data":"a54c93ee1391f792d4dd1bd959360862d369182ab54f972c15d27052f9adc897"} Jul 31 23:46:55.366080 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.366048 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:55.366276 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.366048 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:55.366276 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.366101 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:55.366276 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:55.366195 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbgxq" podUID="9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199" Jul 31 23:46:55.366448 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:55.366330 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlkds" podUID="49904e8b-d888-442a-bab8-1e3c3f83d2e7" Jul 31 23:46:55.366505 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:55.366441 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9sh5" podUID="03fa2c21-f2f9-450a-ac02-5a0de6eef3ee" Jul 31 23:46:55.387267 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.387237 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:55.387543 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.387486 2582 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 31 23:46:55.388022 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.387965 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:55.388129 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.388043 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:55.398831 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.398788 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" podUID="80ad104b-689c-49ca-ad79-0a682d464661" containerName="ovnkube-controller" probeResult="failure" output="" Jul 31 23:46:55.406755 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.406728 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:55.407311 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.407292 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:55.408992 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.408964 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" podUID="80ad104b-689c-49ca-ad79-0a682d464661" containerName="ovnkube-controller" probeResult="failure" output="" Jul 31 23:46:55.503513 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:55.503473 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" podUID="80ad104b-689c-49ca-ad79-0a682d464661" containerName="ovnkube-controller" probeResult="failure" output="" Jul 31 23:46:56.506092 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:56.506063 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jr47x" Jul 31 23:46:56.998685 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:56.998646 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:56.998864 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:56.998700 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:56.998864 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:56.998744 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86t8n\" (UniqueName: \"kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n\") pod \"network-check-target-wlkds\" (UID: \"49904e8b-d888-442a-bab8-1e3c3f83d2e7\") " pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:56.998864 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:56.998770 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:56.998864 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:56.998841 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs podName:9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:28.998815126 +0000 UTC m=+66.158664238 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs") pod "network-metrics-daemon-dbgxq" (UID: "9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 23:46:56.998864 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:56.998858 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 23:46:56.999054 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:56.998873 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 23:46:56.999054 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:56.998877 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:56.999054 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:56.998949 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret podName:03fa2c21-f2f9-450a-ac02-5a0de6eef3ee nodeName:}" failed. No retries permitted until 2026-07-31 23:47:28.998927674 +0000 UTC m=+66.158776781 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret") pod "global-pull-secret-syncer-l9sh5" (UID: "03fa2c21-f2f9-450a-ac02-5a0de6eef3ee") : object "kube-system"/"original-pull-secret" not registered Jul 31 23:46:56.999054 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:56.998885 2582 projected.go:196] Error preparing data for projected volume kube-api-access-86t8n for pod openshift-network-diagnostics/network-check-target-wlkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:56.999054 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:56.999004 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n podName:49904e8b-d888-442a-bab8-1e3c3f83d2e7 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:28.998993718 +0000 UTC m=+66.158842811 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-86t8n" (UniqueName: "kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n") pod "network-check-target-wlkds" (UID: "49904e8b-d888-442a-bab8-1e3c3f83d2e7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 23:46:57.142844 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.142814 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-119.ec2.internal" event="NodeReady" Jul 31 23:46:57.142995 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.142973 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 31 23:46:57.188291 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.188252 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jz6nv"] Jul 31 23:46:57.190516 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.190478 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-frqxm"] Jul 31 23:46:57.190645 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.190608 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jz6nv" Jul 31 23:46:57.194787 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.194764 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 31 23:46:57.195012 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.194993 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-m6h4p\"" Jul 31 23:46:57.195096 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.195036 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 31 23:46:57.195147 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.194996 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 31 23:46:57.195485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.195467 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.197525 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.197509 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 31 23:46:57.197699 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.197571 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rs996\"" Jul 31 23:46:57.197777 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.197617 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 31 23:46:57.199211 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.199165 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-frqxm"] Jul 31 23:46:57.202338 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.202319 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jz6nv"] Jul 31 23:46:57.301378 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.301300 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-config-volume\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.301378 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.301337 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.301580 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.301455 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf-cert\") pod \"ingress-canary-jz6nv\" (UID: \"16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf\") " pod="openshift-ingress-canary/ingress-canary-jz6nv" Jul 31 23:46:57.301580 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.301506 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-tmp-dir\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.301580 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.301539 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsck7\" (UniqueName: \"kubernetes.io/projected/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-kube-api-access-qsck7\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.301580 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.301564 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4455g\" (UniqueName: \"kubernetes.io/projected/16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf-kube-api-access-4455g\") pod \"ingress-canary-jz6nv\" (UID: \"16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf\") " pod="openshift-ingress-canary/ingress-canary-jz6nv" Jul 31 23:46:57.366779 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.366740 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:46:57.366964 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.366740 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:46:57.366964 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.366740 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:46:57.369723 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.369699 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 31 23:46:57.369723 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.369717 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v7qx8\"" Jul 31 23:46:57.369888 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.369714 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 31 23:46:57.369888 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.369743 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 31 23:46:57.370021 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.370008 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 31 23:46:57.370021 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.370016 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5t5tm\"" Jul 31 23:46:57.402628 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.402603 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsck7\" (UniqueName: \"kubernetes.io/projected/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-kube-api-access-qsck7\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.402736 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.402635 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4455g\" (UniqueName: \"kubernetes.io/projected/16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf-kube-api-access-4455g\") pod \"ingress-canary-jz6nv\" (UID: \"16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf\") " pod="openshift-ingress-canary/ingress-canary-jz6nv" Jul 31 23:46:57.402736 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.402707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-config-volume\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.402736 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.402729 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.402900 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.402774 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf-cert\") pod \"ingress-canary-jz6nv\" (UID: \"16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf\") " pod="openshift-ingress-canary/ingress-canary-jz6nv" Jul 31 23:46:57.402900 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.402809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-tmp-dir\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.402991 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:57.402966 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 23:46:57.403042 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:57.403027 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls podName:1481c3cd-a8e1-4d75-a73a-f675c5b936a0 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:57.903007497 +0000 UTC m=+35.062856599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls") pod "dns-default-frqxm" (UID: "1481c3cd-a8e1-4d75-a73a-f675c5b936a0") : secret "dns-default-metrics-tls" not found Jul 31 23:46:57.403134 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.403107 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-tmp-dir\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.403340 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.403318 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-config-volume\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.407474 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.407453 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf-cert\") pod \"ingress-canary-jz6nv\" (UID: \"16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf\") " pod="openshift-ingress-canary/ingress-canary-jz6nv" Jul 31 23:46:57.411375 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.411352 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4455g\" (UniqueName: \"kubernetes.io/projected/16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf-kube-api-access-4455g\") pod \"ingress-canary-jz6nv\" (UID: \"16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf\") " pod="openshift-ingress-canary/ingress-canary-jz6nv" Jul 31 23:46:57.411495 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.411351 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsck7\" (UniqueName: \"kubernetes.io/projected/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-kube-api-access-qsck7\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.505308 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.505280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jz6nv" Jul 31 23:46:57.906704 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:57.906669 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:57.907263 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:57.906855 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 23:46:57.907263 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:57.906932 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls podName:1481c3cd-a8e1-4d75-a73a-f675c5b936a0 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:58.906911878 +0000 UTC m=+36.066760970 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls") pod "dns-default-frqxm" (UID: "1481c3cd-a8e1-4d75-a73a-f675c5b936a0") : secret "dns-default-metrics-tls" not found Jul 31 23:46:58.405062 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:58.404903 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jz6nv"] Jul 31 23:46:58.409397 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:46:58.409366 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16a41c32_9c4a_4ea7_9c1a_9aeb6f9b33cf.slice/crio-5322527f683b2b0d7d59523a9678bd7d377b9fe0dfb4796e6a96f6cd43331679 WatchSource:0}: Error finding container 5322527f683b2b0d7d59523a9678bd7d377b9fe0dfb4796e6a96f6cd43331679: Status 404 returned error can't find the container with id 5322527f683b2b0d7d59523a9678bd7d377b9fe0dfb4796e6a96f6cd43331679 Jul 31 23:46:58.503909 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:58.503881 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smx29" event={"ID":"76114bf1-387d-413b-8b61-9cd4d084d3d9","Type":"ContainerStarted","Data":"8ff4a2c28f716a74d3d71e51b7426330321a6a2e317b863c8039a288b967b711"} Jul 31 23:46:58.505538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:58.505516 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jz6nv" event={"ID":"16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf","Type":"ContainerStarted","Data":"5322527f683b2b0d7d59523a9678bd7d377b9fe0dfb4796e6a96f6cd43331679"} Jul 31 23:46:58.915357 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:58.915312 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:46:58.915911 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:58.915512 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 23:46:58.915911 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:58.915611 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls podName:1481c3cd-a8e1-4d75-a73a-f675c5b936a0 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:00.915585149 +0000 UTC m=+38.075434241 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls") pod "dns-default-frqxm" (UID: "1481c3cd-a8e1-4d75-a73a-f675c5b936a0") : secret "dns-default-metrics-tls" not found Jul 31 23:46:59.511261 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.511230 2582 generic.go:356] "Generic (PLEG): container finished" podID="76114bf1-387d-413b-8b61-9cd4d084d3d9" containerID="8ff4a2c28f716a74d3d71e51b7426330321a6a2e317b863c8039a288b967b711" exitCode=0 Jul 31 23:46:59.511493 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.511295 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smx29" event={"ID":"76114bf1-387d-413b-8b61-9cd4d084d3d9","Type":"ContainerDied","Data":"8ff4a2c28f716a74d3d71e51b7426330321a6a2e317b863c8039a288b967b711"} Jul 31 23:46:59.588303 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.588268 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5xm8p"] Jul 31 23:46:59.591009 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.590987 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.593395 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.593367 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 31 23:46:59.593395 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.593395 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 31 23:46:59.593726 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.593708 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 31 23:46:59.593814 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.593785 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fst58\"" Jul 31 23:46:59.593814 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.593800 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 31 23:46:59.600262 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.600240 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5xm8p"] Jul 31 23:46:59.721579 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.721541 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.721579 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.721587 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kpc4\" (UniqueName: \"kubernetes.io/projected/053b510c-cca6-4383-84b2-836a9d686abc-kube-api-access-8kpc4\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.721802 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.721667 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/053b510c-cca6-4383-84b2-836a9d686abc-crio-socket\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.721802 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.721751 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.721802 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.721783 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/053b510c-cca6-4383-84b2-836a9d686abc-data-volume\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.822350 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.822258 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/053b510c-cca6-4383-84b2-836a9d686abc-data-volume\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.822532 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.822383 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.822532 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.822429 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kpc4\" (UniqueName: \"kubernetes.io/projected/053b510c-cca6-4383-84b2-836a9d686abc-kube-api-access-8kpc4\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.822532 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.822482 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/053b510c-cca6-4383-84b2-836a9d686abc-crio-socket\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.822532 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.822508 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.822532 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:59.822511 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 23:46:59.822766 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:59.822602 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm podName:053b510c-cca6-4383-84b2-836a9d686abc nodeName:}" failed. No retries permitted until 2026-07-31 23:47:00.322582297 +0000 UTC m=+37.482431406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5xm8p" (UID: "053b510c-cca6-4383-84b2-836a9d686abc") : configmap "kube-rbac-proxy" not found Jul 31 23:46:59.822766 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:59.822611 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 23:46:59.822766 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:46:59.822660 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls podName:053b510c-cca6-4383-84b2-836a9d686abc nodeName:}" failed. No retries permitted until 2026-07-31 23:47:00.322645317 +0000 UTC m=+37.482494420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5xm8p" (UID: "053b510c-cca6-4383-84b2-836a9d686abc") : secret "insights-runtime-extractor-tls" not found Jul 31 23:46:59.822766 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.822668 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/053b510c-cca6-4383-84b2-836a9d686abc-data-volume\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.822944 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.822823 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/053b510c-cca6-4383-84b2-836a9d686abc-crio-socket\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.838145 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.838116 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kpc4\" (UniqueName: \"kubernetes.io/projected/053b510c-cca6-4383-84b2-836a9d686abc-kube-api-access-8kpc4\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:46:59.895713 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.895676 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn"] Jul 31 23:46:59.918234 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.918194 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn"] Jul 31 23:46:59.918684 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.918305 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn" Jul 31 23:46:59.920934 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.920908 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 31 23:46:59.921042 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.920919 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 31 23:46:59.921256 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:46:59.921234 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-glv7n\"" Jul 31 23:47:00.024183 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:00.024109 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqbg9\" (UniqueName: \"kubernetes.io/projected/85de7af4-868a-4913-983b-5691cc557068-kube-api-access-hqbg9\") pod \"migrator-68bc745d87-47cbn\" (UID: \"85de7af4-868a-4913-983b-5691cc557068\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn" Jul 31 23:47:00.124853 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:00.124754 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqbg9\" (UniqueName: \"kubernetes.io/projected/85de7af4-868a-4913-983b-5691cc557068-kube-api-access-hqbg9\") pod \"migrator-68bc745d87-47cbn\" (UID: \"85de7af4-868a-4913-983b-5691cc557068\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn" Jul 31 23:47:00.136130 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:00.136091 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqbg9\" (UniqueName: \"kubernetes.io/projected/85de7af4-868a-4913-983b-5691cc557068-kube-api-access-hqbg9\") pod \"migrator-68bc745d87-47cbn\" (UID: \"85de7af4-868a-4913-983b-5691cc557068\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn" Jul 31 23:47:00.228724 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:00.228690 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn" Jul 31 23:47:00.326553 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:00.326306 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:00.326721 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:00.326589 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:00.326721 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:00.326474 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 23:47:00.326721 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:00.326703 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm podName:053b510c-cca6-4383-84b2-836a9d686abc nodeName:}" failed. No retries permitted until 2026-07-31 23:47:01.326680715 +0000 UTC m=+38.486529819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5xm8p" (UID: "053b510c-cca6-4383-84b2-836a9d686abc") : configmap "kube-rbac-proxy" not found Jul 31 23:47:00.326721 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:00.326714 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 23:47:00.326954 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:00.326749 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls podName:053b510c-cca6-4383-84b2-836a9d686abc nodeName:}" failed. No retries permitted until 2026-07-31 23:47:01.326738988 +0000 UTC m=+38.486588083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5xm8p" (UID: "053b510c-cca6-4383-84b2-836a9d686abc") : secret "insights-runtime-extractor-tls" not found Jul 31 23:47:00.514501 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:00.514466 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn"] Jul 31 23:47:00.517757 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:00.517734 2582 generic.go:356] "Generic (PLEG): container finished" podID="76114bf1-387d-413b-8b61-9cd4d084d3d9" containerID="99223303e863ccc35b965fe27ae273eef36fedae7c6b9274b9d18ed223fea589" exitCode=0 Jul 31 23:47:00.517852 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:00.517787 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smx29" event={"ID":"76114bf1-387d-413b-8b61-9cd4d084d3d9","Type":"ContainerDied","Data":"99223303e863ccc35b965fe27ae273eef36fedae7c6b9274b9d18ed223fea589"} Jul 31 23:47:00.518283 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:00.518254 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85de7af4_868a_4913_983b_5691cc557068.slice/crio-c601b391aeba40a033a8d1c373463110700676afe4b638b4d8b474d147f2d9fc WatchSource:0}: Error finding container c601b391aeba40a033a8d1c373463110700676afe4b638b4d8b474d147f2d9fc: Status 404 returned error can't find the container with id c601b391aeba40a033a8d1c373463110700676afe4b638b4d8b474d147f2d9fc Jul 31 23:47:00.930688 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:00.930603 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:47:00.931432 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:00.930735 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 23:47:00.931432 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:00.930794 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls podName:1481c3cd-a8e1-4d75-a73a-f675c5b936a0 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:04.930780163 +0000 UTC m=+42.090629252 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls") pod "dns-default-frqxm" (UID: "1481c3cd-a8e1-4d75-a73a-f675c5b936a0") : secret "dns-default-metrics-tls" not found Jul 31 23:47:01.333881 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:01.333841 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:01.334091 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:01.333905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:01.334091 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:01.333988 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 23:47:01.334091 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:01.334051 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 23:47:01.334091 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:01.334078 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm podName:053b510c-cca6-4383-84b2-836a9d686abc nodeName:}" failed. No retries permitted until 2026-07-31 23:47:03.334055666 +0000 UTC m=+40.493904759 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5xm8p" (UID: "053b510c-cca6-4383-84b2-836a9d686abc") : configmap "kube-rbac-proxy" not found Jul 31 23:47:01.334258 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:01.334113 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls podName:053b510c-cca6-4383-84b2-836a9d686abc nodeName:}" failed. No retries permitted until 2026-07-31 23:47:03.334096867 +0000 UTC m=+40.493945956 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5xm8p" (UID: "053b510c-cca6-4383-84b2-836a9d686abc") : secret "insights-runtime-extractor-tls" not found Jul 31 23:47:01.444611 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:01.444587 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6czd7_16ff1b3d-8225-4e5e-a2df-3ea9b0205971/dns-node-resolver/0.log" Jul 31 23:47:01.523763 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:01.523726 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smx29" event={"ID":"76114bf1-387d-413b-8b61-9cd4d084d3d9","Type":"ContainerStarted","Data":"288ac73566225c849e6846dbc7b9b3b82b1fc0774a616a433001e512bb9e5c5b"} Jul 31 23:47:01.525143 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:01.525115 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jz6nv" event={"ID":"16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf","Type":"ContainerStarted","Data":"4eafb1a710e939cd46c9cdb2b667119c2717c622d4b90a20e9d6e2905e6da0dd"} Jul 31 23:47:01.526098 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:01.526081 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn" event={"ID":"85de7af4-868a-4913-983b-5691cc557068","Type":"ContainerStarted","Data":"c601b391aeba40a033a8d1c373463110700676afe4b638b4d8b474d147f2d9fc"} Jul 31 23:47:01.546517 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:01.546473 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-smx29" podStartSLOduration=10.465160788 podStartE2EDuration="38.546457441s" totalImagesPullingTime="28.081296653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:01.544685981 +0000 UTC m=+38.704535094" watchObservedRunningTime="2026-07-31 23:47:01.546457441 +0000 UTC m=+38.706306530" Jul 31 23:47:01.560115 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:01.560074 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jz6nv" podStartSLOduration=2.5036562719999997 podStartE2EDuration="4.560059879s" totalImagesPullingTime="2.056403607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:01.559052456 +0000 UTC m=+38.718901568" watchObservedRunningTime="2026-07-31 23:47:01.560059879 +0000 UTC m=+38.719908969" Jul 31 23:47:02.137127 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.137033 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-9ns9t"] Jul 31 23:47:02.158841 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.158818 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" Jul 31 23:47:02.161343 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.161318 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-9ns9t"] Jul 31 23:47:02.161654 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.161623 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 31 23:47:02.162083 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.162066 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 31 23:47:02.162160 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.162144 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 31 23:47:02.163163 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.163143 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 31 23:47:02.163256 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.163203 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-q87ck\"" Jul 31 23:47:02.241200 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.241169 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b30515f9-283a-403d-b427-6680bf93c45b-signing-key\") pod \"service-ca-98c88fbb5-9ns9t\" (UID: \"b30515f9-283a-403d-b427-6680bf93c45b\") " pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" Jul 31 23:47:02.241361 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.241223 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b30515f9-283a-403d-b427-6680bf93c45b-signing-cabundle\") pod \"service-ca-98c88fbb5-9ns9t\" (UID: \"b30515f9-283a-403d-b427-6680bf93c45b\") " pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" Jul 31 23:47:02.241361 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.241252 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjw4v\" (UniqueName: \"kubernetes.io/projected/b30515f9-283a-403d-b427-6680bf93c45b-kube-api-access-zjw4v\") pod \"service-ca-98c88fbb5-9ns9t\" (UID: \"b30515f9-283a-403d-b427-6680bf93c45b\") " pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" Jul 31 23:47:02.342385 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.342351 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b30515f9-283a-403d-b427-6680bf93c45b-signing-key\") pod \"service-ca-98c88fbb5-9ns9t\" (UID: \"b30515f9-283a-403d-b427-6680bf93c45b\") " pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" Jul 31 23:47:02.342559 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.342429 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b30515f9-283a-403d-b427-6680bf93c45b-signing-cabundle\") pod \"service-ca-98c88fbb5-9ns9t\" (UID: \"b30515f9-283a-403d-b427-6680bf93c45b\") " pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" Jul 31 23:47:02.342559 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.342466 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjw4v\" (UniqueName: \"kubernetes.io/projected/b30515f9-283a-403d-b427-6680bf93c45b-kube-api-access-zjw4v\") pod \"service-ca-98c88fbb5-9ns9t\" (UID: \"b30515f9-283a-403d-b427-6680bf93c45b\") " pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" Jul 31 23:47:02.343022 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.343003 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b30515f9-283a-403d-b427-6680bf93c45b-signing-cabundle\") pod \"service-ca-98c88fbb5-9ns9t\" (UID: \"b30515f9-283a-403d-b427-6680bf93c45b\") " pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" Jul 31 23:47:02.346144 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.346126 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b30515f9-283a-403d-b427-6680bf93c45b-signing-key\") pod \"service-ca-98c88fbb5-9ns9t\" (UID: \"b30515f9-283a-403d-b427-6680bf93c45b\") " pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" Jul 31 23:47:02.350541 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.350517 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjw4v\" (UniqueName: \"kubernetes.io/projected/b30515f9-283a-403d-b427-6680bf93c45b-kube-api-access-zjw4v\") pod \"service-ca-98c88fbb5-9ns9t\" (UID: \"b30515f9-283a-403d-b427-6680bf93c45b\") " pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" Jul 31 23:47:02.468831 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.468746 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" Jul 31 23:47:02.530007 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.529971 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn" event={"ID":"85de7af4-868a-4913-983b-5691cc557068","Type":"ContainerStarted","Data":"ad7d38d150508e554eeb03eedcae51f1227456292884476408ca392c76951ba8"} Jul 31 23:47:02.530168 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.530026 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn" event={"ID":"85de7af4-868a-4913-983b-5691cc557068","Type":"ContainerStarted","Data":"40b06ea4caf4fe530087957db6c2f02be3b52702e877a3756b7765f97a25f020"} Jul 31 23:47:02.545672 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.545609 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-47cbn" podStartSLOduration=2.229504295 podStartE2EDuration="3.545588818s" totalImagesPullingTime="1.316084523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:02.54518789 +0000 UTC m=+39.705037003" watchObservedRunningTime="2026-07-31 23:47:02.545588818 +0000 UTC m=+39.705437931" Jul 31 23:47:02.614843 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.614814 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-9ns9t"] Jul 31 23:47:02.618342 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:02.618317 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb30515f9_283a_403d_b427_6680bf93c45b.slice/crio-f91bbeca1e304f0ea57e1b19147bcda475047b8e23858911c85c7c64b193d802 WatchSource:0}: Error finding container f91bbeca1e304f0ea57e1b19147bcda475047b8e23858911c85c7c64b193d802: Status 404 returned error can't find the container with id f91bbeca1e304f0ea57e1b19147bcda475047b8e23858911c85c7c64b193d802 Jul 31 23:47:02.847372 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:02.847345 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qwln9_2262118f-b73c-4dbf-9ca6-d14c05495100/node-ca/0.log" Jul 31 23:47:03.351075 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:03.351042 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:03.351496 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:03.351104 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:03.351496 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:03.351151 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 23:47:03.351496 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:03.351232 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm podName:053b510c-cca6-4383-84b2-836a9d686abc nodeName:}" failed. No retries permitted until 2026-07-31 23:47:07.35121413 +0000 UTC m=+44.511063221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5xm8p" (UID: "053b510c-cca6-4383-84b2-836a9d686abc") : configmap "kube-rbac-proxy" not found Jul 31 23:47:03.351496 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:03.351234 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 23:47:03.351496 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:03.351299 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls podName:053b510c-cca6-4383-84b2-836a9d686abc nodeName:}" failed. No retries permitted until 2026-07-31 23:47:07.351269991 +0000 UTC m=+44.511119083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5xm8p" (UID: "053b510c-cca6-4383-84b2-836a9d686abc") : secret "insights-runtime-extractor-tls" not found Jul 31 23:47:03.534379 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:03.534336 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" event={"ID":"b30515f9-283a-403d-b427-6680bf93c45b","Type":"ContainerStarted","Data":"f91bbeca1e304f0ea57e1b19147bcda475047b8e23858911c85c7c64b193d802"} Jul 31 23:47:03.644740 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:03.644668 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jz6nv_16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf/serve-healthcheck-canary/0.log" Jul 31 23:47:04.964550 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:04.964465 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:47:04.964989 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:04.964654 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 23:47:04.964989 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:04.964744 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls podName:1481c3cd-a8e1-4d75-a73a-f675c5b936a0 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:12.964720542 +0000 UTC m=+50.124569646 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls") pod "dns-default-frqxm" (UID: "1481c3cd-a8e1-4d75-a73a-f675c5b936a0") : secret "dns-default-metrics-tls" not found Jul 31 23:47:05.540722 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:05.540685 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" event={"ID":"b30515f9-283a-403d-b427-6680bf93c45b","Type":"ContainerStarted","Data":"88db68d034a449301bfea0fb509a94625957ac30e861ec12cb79457cc0ebe46f"} Jul 31 23:47:05.558313 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:05.558103 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-9ns9t" podStartSLOduration=1.6046863710000001 podStartE2EDuration="3.55808508s" totalImagesPullingTime="1.953398709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:05.557605228 +0000 UTC m=+42.717454352" watchObservedRunningTime="2026-07-31 23:47:05.55808508 +0000 UTC m=+42.717934193" Jul 31 23:47:07.384325 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:07.384291 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:07.384812 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:07.384385 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:07.384812 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:07.384483 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 23:47:07.384812 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:07.384591 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls podName:053b510c-cca6-4383-84b2-836a9d686abc nodeName:}" failed. No retries permitted until 2026-07-31 23:47:15.384543668 +0000 UTC m=+52.544392781 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5xm8p" (UID: "053b510c-cca6-4383-84b2-836a9d686abc") : secret "insights-runtime-extractor-tls" not found Jul 31 23:47:07.384930 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:07.384878 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/053b510c-cca6-4383-84b2-836a9d686abc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:13.026693 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:13.026645 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:47:13.028989 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:13.028963 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1481c3cd-a8e1-4d75-a73a-f675c5b936a0-metrics-tls\") pod \"dns-default-frqxm\" (UID: \"1481c3cd-a8e1-4d75-a73a-f675c5b936a0\") " pod="openshift-dns/dns-default-frqxm" Jul 31 23:47:13.111363 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:13.111323 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-frqxm" Jul 31 23:47:13.269483 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:13.269452 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-frqxm"] Jul 31 23:47:13.272102 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:13.272074 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1481c3cd_a8e1_4d75_a73a_f675c5b936a0.slice/crio-5602ac80e19afd55fc0f12399753cb7a370db2c4c7db0483d2ede05055d6a602 WatchSource:0}: Error finding container 5602ac80e19afd55fc0f12399753cb7a370db2c4c7db0483d2ede05055d6a602: Status 404 returned error can't find the container with id 5602ac80e19afd55fc0f12399753cb7a370db2c4c7db0483d2ede05055d6a602 Jul 31 23:47:13.564583 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:13.564549 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-frqxm" event={"ID":"1481c3cd-a8e1-4d75-a73a-f675c5b936a0","Type":"ContainerStarted","Data":"5602ac80e19afd55fc0f12399753cb7a370db2c4c7db0483d2ede05055d6a602"} Jul 31 23:47:15.445455 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:15.445399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:15.445842 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:15.445518 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 23:47:15.445842 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:15.445590 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls podName:053b510c-cca6-4383-84b2-836a9d686abc nodeName:}" failed. No retries permitted until 2026-07-31 23:47:31.445574069 +0000 UTC m=+68.605423162 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5xm8p" (UID: "053b510c-cca6-4383-84b2-836a9d686abc") : secret "insights-runtime-extractor-tls" not found Jul 31 23:47:15.572693 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:15.572655 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-frqxm" event={"ID":"1481c3cd-a8e1-4d75-a73a-f675c5b936a0","Type":"ContainerStarted","Data":"a6da21d78e03a923579c2713d5e5f3df5c671422ad7ab2dd395482532c3bb1bc"} Jul 31 23:47:15.572898 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:15.572713 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-frqxm" event={"ID":"1481c3cd-a8e1-4d75-a73a-f675c5b936a0","Type":"ContainerStarted","Data":"a77d527a65c5d531c8f15899e25903316fc30e3daf113abf7c3c0b357dca4277"} Jul 31 23:47:15.572898 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:15.572757 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-frqxm" Jul 31 23:47:15.596332 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:15.596284 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-frqxm" podStartSLOduration=16.779921499 podStartE2EDuration="18.59626885s" totalImagesPullingTime="1.816347351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:15.595589979 +0000 UTC m=+52.755439090" watchObservedRunningTime="2026-07-31 23:47:15.59626885 +0000 UTC m=+52.756117985" Jul 31 23:47:22.365531 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.365500 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896"] Jul 31 23:47:22.396397 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.396369 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896"] Jul 31 23:47:22.396562 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.396487 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896" Jul 31 23:47:22.399061 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.399034 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 31 23:47:22.400015 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.399989 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hh5ps\"" Jul 31 23:47:22.400117 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.400017 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 31 23:47:22.400233 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.400219 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 31 23:47:22.400330 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.400313 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 31 23:47:22.469753 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.469714 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-6qbq8"] Jul 31 23:47:22.490699 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.490662 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-659bffd457-92tkj"] Jul 31 23:47:22.490894 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.490783 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-6qbq8" Jul 31 23:47:22.493458 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.493437 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 31 23:47:22.493563 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.493514 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ndcrg\"" Jul 31 23:47:22.493563 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.493438 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 31 23:47:22.496864 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.496835 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdgtm\" (UniqueName: \"kubernetes.io/projected/d57d11a1-0e56-4450-be31-cb35ddb0d7ea-kube-api-access-kdgtm\") pod \"managed-serviceaccount-addon-agent-6d45f9f6d6-6k896\" (UID: \"d57d11a1-0e56-4450-be31-cb35ddb0d7ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896" Jul 31 23:47:22.496972 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.496891 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d57d11a1-0e56-4450-be31-cb35ddb0d7ea-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d45f9f6d6-6k896\" (UID: \"d57d11a1-0e56-4450-be31-cb35ddb0d7ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896" Jul 31 23:47:22.512933 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.512910 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-6qbq8"] Jul 31 23:47:22.513027 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.512939 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-659bffd457-92tkj"] Jul 31 23:47:22.513027 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.513020 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.515354 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.515336 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 31 23:47:22.515512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.515340 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fxfp4\"" Jul 31 23:47:22.515645 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.515628 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 31 23:47:22.515734 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.515717 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 31 23:47:22.520703 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.520685 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 31 23:47:22.597611 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.597579 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/065b662e-d891-4fd5-a2a9-b610d1aa991a-image-registry-private-configuration\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.597611 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.597611 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/065b662e-d891-4fd5-a2a9-b610d1aa991a-registry-certificates\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.597821 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.597631 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-675hs\" (UniqueName: \"kubernetes.io/projected/065b662e-d891-4fd5-a2a9-b610d1aa991a-kube-api-access-675hs\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.597821 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.597714 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/065b662e-d891-4fd5-a2a9-b610d1aa991a-registry-tls\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.597821 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.597759 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdgtm\" (UniqueName: \"kubernetes.io/projected/d57d11a1-0e56-4450-be31-cb35ddb0d7ea-kube-api-access-kdgtm\") pod \"managed-serviceaccount-addon-agent-6d45f9f6d6-6k896\" (UID: \"d57d11a1-0e56-4450-be31-cb35ddb0d7ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896" Jul 31 23:47:22.597821 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.597778 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/065b662e-d891-4fd5-a2a9-b610d1aa991a-ca-trust-extracted\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.597821 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.597798 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/065b662e-d891-4fd5-a2a9-b610d1aa991a-trusted-ca\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.597821 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.597815 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/065b662e-d891-4fd5-a2a9-b610d1aa991a-bound-sa-token\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.598083 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.597865 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/065b662e-d891-4fd5-a2a9-b610d1aa991a-installation-pull-secrets\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.598083 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.597891 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d57d11a1-0e56-4450-be31-cb35ddb0d7ea-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d45f9f6d6-6k896\" (UID: \"d57d11a1-0e56-4450-be31-cb35ddb0d7ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896" Jul 31 23:47:22.598083 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.597909 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzrwz\" (UniqueName: \"kubernetes.io/projected/8b00e295-a506-43f7-b005-3de7ed37ea94-kube-api-access-tzrwz\") pod \"downloads-77c974ffcf-6qbq8\" (UID: \"8b00e295-a506-43f7-b005-3de7ed37ea94\") " pod="openshift-console/downloads-77c974ffcf-6qbq8" Jul 31 23:47:22.600230 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.600210 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d57d11a1-0e56-4450-be31-cb35ddb0d7ea-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d45f9f6d6-6k896\" (UID: \"d57d11a1-0e56-4450-be31-cb35ddb0d7ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896" Jul 31 23:47:22.608254 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.608229 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdgtm\" (UniqueName: \"kubernetes.io/projected/d57d11a1-0e56-4450-be31-cb35ddb0d7ea-kube-api-access-kdgtm\") pod \"managed-serviceaccount-addon-agent-6d45f9f6d6-6k896\" (UID: \"d57d11a1-0e56-4450-be31-cb35ddb0d7ea\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896" Jul 31 23:47:22.698314 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.698233 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/065b662e-d891-4fd5-a2a9-b610d1aa991a-installation-pull-secrets\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.698314 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.698273 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzrwz\" (UniqueName: \"kubernetes.io/projected/8b00e295-a506-43f7-b005-3de7ed37ea94-kube-api-access-tzrwz\") pod \"downloads-77c974ffcf-6qbq8\" (UID: \"8b00e295-a506-43f7-b005-3de7ed37ea94\") " pod="openshift-console/downloads-77c974ffcf-6qbq8" Jul 31 23:47:22.698573 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.698326 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/065b662e-d891-4fd5-a2a9-b610d1aa991a-image-registry-private-configuration\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.698573 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.698349 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/065b662e-d891-4fd5-a2a9-b610d1aa991a-registry-certificates\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.698573 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.698371 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-675hs\" (UniqueName: \"kubernetes.io/projected/065b662e-d891-4fd5-a2a9-b610d1aa991a-kube-api-access-675hs\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.698721 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.698583 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/065b662e-d891-4fd5-a2a9-b610d1aa991a-registry-tls\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.698721 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.698638 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/065b662e-d891-4fd5-a2a9-b610d1aa991a-ca-trust-extracted\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.698721 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.698678 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/065b662e-d891-4fd5-a2a9-b610d1aa991a-trusted-ca\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.698721 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.698708 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/065b662e-d891-4fd5-a2a9-b610d1aa991a-bound-sa-token\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.699084 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.699060 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/065b662e-d891-4fd5-a2a9-b610d1aa991a-ca-trust-extracted\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.699264 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.699248 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/065b662e-d891-4fd5-a2a9-b610d1aa991a-registry-certificates\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.699742 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.699718 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/065b662e-d891-4fd5-a2a9-b610d1aa991a-trusted-ca\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.701042 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.701019 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/065b662e-d891-4fd5-a2a9-b610d1aa991a-installation-pull-secrets\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.701358 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.701338 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/065b662e-d891-4fd5-a2a9-b610d1aa991a-image-registry-private-configuration\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.701430 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.701343 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/065b662e-d891-4fd5-a2a9-b610d1aa991a-registry-tls\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.710046 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.710019 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzrwz\" (UniqueName: \"kubernetes.io/projected/8b00e295-a506-43f7-b005-3de7ed37ea94-kube-api-access-tzrwz\") pod \"downloads-77c974ffcf-6qbq8\" (UID: \"8b00e295-a506-43f7-b005-3de7ed37ea94\") " pod="openshift-console/downloads-77c974ffcf-6qbq8" Jul 31 23:47:22.710165 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.710111 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/065b662e-d891-4fd5-a2a9-b610d1aa991a-bound-sa-token\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.710915 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.710896 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-675hs\" (UniqueName: \"kubernetes.io/projected/065b662e-d891-4fd5-a2a9-b610d1aa991a-kube-api-access-675hs\") pod \"image-registry-659bffd457-92tkj\" (UID: \"065b662e-d891-4fd5-a2a9-b610d1aa991a\") " pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.716903 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.716881 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896" Jul 31 23:47:22.799506 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.799237 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-6qbq8" Jul 31 23:47:22.820911 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.820879 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:22.867544 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.867479 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896"] Jul 31 23:47:22.872831 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:22.872799 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd57d11a1_0e56_4450_be31_cb35ddb0d7ea.slice/crio-cdef136d57f915ceb23fb2a3858f13121abc047aa4ddaffc6bef45584c83427d WatchSource:0}: Error finding container cdef136d57f915ceb23fb2a3858f13121abc047aa4ddaffc6bef45584c83427d: Status 404 returned error can't find the container with id cdef136d57f915ceb23fb2a3858f13121abc047aa4ddaffc6bef45584c83427d Jul 31 23:47:22.940487 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.940455 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-6qbq8"] Jul 31 23:47:22.943096 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:22.943069 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b00e295_a506_43f7_b005_3de7ed37ea94.slice/crio-db32d92345a9a7f4c9bc2720b85870cec650550c7b1182e0a5bdfb1b2db36612 WatchSource:0}: Error finding container db32d92345a9a7f4c9bc2720b85870cec650550c7b1182e0a5bdfb1b2db36612: Status 404 returned error can't find the container with id db32d92345a9a7f4c9bc2720b85870cec650550c7b1182e0a5bdfb1b2db36612 Jul 31 23:47:22.965933 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:22.965871 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-659bffd457-92tkj"] Jul 31 23:47:22.978396 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:22.978369 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod065b662e_d891_4fd5_a2a9_b610d1aa991a.slice/crio-8df0028265044594b319195eb8df0180179d14283701daa8da2fdc5627e6e1a2 WatchSource:0}: Error finding container 8df0028265044594b319195eb8df0180179d14283701daa8da2fdc5627e6e1a2: Status 404 returned error can't find the container with id 8df0028265044594b319195eb8df0180179d14283701daa8da2fdc5627e6e1a2 Jul 31 23:47:23.597792 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:23.597715 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-6qbq8" event={"ID":"8b00e295-a506-43f7-b005-3de7ed37ea94","Type":"ContainerStarted","Data":"db32d92345a9a7f4c9bc2720b85870cec650550c7b1182e0a5bdfb1b2db36612"} Jul 31 23:47:23.600015 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:23.599961 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896" event={"ID":"d57d11a1-0e56-4450-be31-cb35ddb0d7ea","Type":"ContainerStarted","Data":"cdef136d57f915ceb23fb2a3858f13121abc047aa4ddaffc6bef45584c83427d"} Jul 31 23:47:23.602352 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:23.602241 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659bffd457-92tkj" event={"ID":"065b662e-d891-4fd5-a2a9-b610d1aa991a","Type":"ContainerStarted","Data":"aeea5ed947cdf06807ce3722c41e89bf7c839ce9ddf5946f0efb751f0d2e9e2c"} Jul 31 23:47:23.602352 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:23.602284 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659bffd457-92tkj" event={"ID":"065b662e-d891-4fd5-a2a9-b610d1aa991a","Type":"ContainerStarted","Data":"8df0028265044594b319195eb8df0180179d14283701daa8da2fdc5627e6e1a2"} Jul 31 23:47:23.602352 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:23.602324 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:23.622572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:23.622519 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-659bffd457-92tkj" podStartSLOduration=1.622499054 podStartE2EDuration="1.622499054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:23.620292355 +0000 UTC m=+60.780141469" watchObservedRunningTime="2026-07-31 23:47:23.622499054 +0000 UTC m=+60.782348165" Jul 31 23:47:25.578164 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:25.578123 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-frqxm" Jul 31 23:47:26.616162 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:26.616125 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896" event={"ID":"d57d11a1-0e56-4450-be31-cb35ddb0d7ea","Type":"ContainerStarted","Data":"1f0fd535838727cf790ef8d16cbdf413c733230c0ccc1434710af5b338cc895a"} Jul 31 23:47:26.632492 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:26.632442 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d45f9f6d6-6k896" podStartSLOduration=1.921179214 podStartE2EDuration="4.632422643s" totalImagesPullingTime="2.711243429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:26.632122066 +0000 UTC m=+63.791971180" watchObservedRunningTime="2026-07-31 23:47:26.632422643 +0000 UTC m=+63.792271747" Jul 31 23:47:27.616182 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.616146 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6859f76475-9q2dc"] Jul 31 23:47:27.618348 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.618320 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.621048 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.621022 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 31 23:47:27.621995 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.621971 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 31 23:47:27.622115 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.622015 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 31 23:47:27.622176 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.622112 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z24jv\"" Jul 31 23:47:27.622176 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.622127 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 31 23:47:27.622275 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.622235 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 31 23:47:27.629671 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.629651 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6859f76475-9q2dc"] Jul 31 23:47:27.640190 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.640165 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnd6n\" (UniqueName: \"kubernetes.io/projected/b27773f2-7e52-4c78-8bac-1c23a9629369-kube-api-access-nnd6n\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.640311 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.640200 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-oauth-config\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.640311 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.640222 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-service-ca\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.640428 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.640353 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-serving-cert\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.640489 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.640437 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-oauth-serving-cert\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.640489 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.640480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-console-config\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.741622 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.741584 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnd6n\" (UniqueName: \"kubernetes.io/projected/b27773f2-7e52-4c78-8bac-1c23a9629369-kube-api-access-nnd6n\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.741622 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.741626 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-oauth-config\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.741839 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.741649 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-service-ca\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.741983 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.741959 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-serving-cert\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.742045 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.742017 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-oauth-serving-cert\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.742089 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.742061 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-console-config\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.742518 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.742473 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-service-ca\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.742733 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.742714 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-oauth-serving-cert\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.742841 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.742820 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-console-config\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.744383 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.744363 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-oauth-config\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.744595 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.744576 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-serving-cert\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.752002 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.751982 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnd6n\" (UniqueName: \"kubernetes.io/projected/b27773f2-7e52-4c78-8bac-1c23a9629369-kube-api-access-nnd6n\") pod \"console-6859f76475-9q2dc\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:27.929489 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:27.929386 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:28.071456 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:28.071426 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6859f76475-9q2dc"] Jul 31 23:47:28.074739 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:28.074710 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb27773f2_7e52_4c78_8bac_1c23a9629369.slice/crio-73a7665f4d1262f587e2cb314da217abe0295d416185ab38a0791bc80c02e5f6 WatchSource:0}: Error finding container 73a7665f4d1262f587e2cb314da217abe0295d416185ab38a0791bc80c02e5f6: Status 404 returned error can't find the container with id 73a7665f4d1262f587e2cb314da217abe0295d416185ab38a0791bc80c02e5f6 Jul 31 23:47:28.629956 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:28.629915 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6859f76475-9q2dc" event={"ID":"b27773f2-7e52-4c78-8bac-1c23a9629369","Type":"ContainerStarted","Data":"73a7665f4d1262f587e2cb314da217abe0295d416185ab38a0791bc80c02e5f6"} Jul 31 23:47:29.051788 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.051746 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86t8n\" (UniqueName: \"kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n\") pod \"network-check-target-wlkds\" (UID: \"49904e8b-d888-442a-bab8-1e3c3f83d2e7\") " pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:47:29.052018 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.051839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:47:29.052018 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.051867 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:47:29.054157 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.054129 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 31 23:47:29.054157 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.054149 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 31 23:47:29.055358 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.055339 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 31 23:47:29.064896 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.064865 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 31 23:47:29.065670 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.065645 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03fa2c21-f2f9-450a-ac02-5a0de6eef3ee-original-pull-secret\") pod \"global-pull-secret-syncer-l9sh5\" (UID: \"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee\") " pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:47:29.066299 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.066277 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199-metrics-certs\") pod \"network-metrics-daemon-dbgxq\" (UID: \"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199\") " pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:47:29.077319 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.077285 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86t8n\" (UniqueName: \"kubernetes.io/projected/49904e8b-d888-442a-bab8-1e3c3f83d2e7-kube-api-access-86t8n\") pod \"network-check-target-wlkds\" (UID: \"49904e8b-d888-442a-bab8-1e3c3f83d2e7\") " pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:47:29.181668 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.181629 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5t5tm\"" Jul 31 23:47:29.185330 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.185090 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9sh5" Jul 31 23:47:29.190266 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.190242 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:47:29.193879 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.193855 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v7qx8\"" Jul 31 23:47:29.202222 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.202186 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbgxq" Jul 31 23:47:29.369082 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.368992 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l9sh5"] Jul 31 23:47:29.380259 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:29.380090 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03fa2c21_f2f9_450a_ac02_5a0de6eef3ee.slice/crio-ac90c756d832eb2c889cdeab38dd6d1a3d419d8aaff060fc5eb46ccf81f1c8c1 WatchSource:0}: Error finding container ac90c756d832eb2c889cdeab38dd6d1a3d419d8aaff060fc5eb46ccf81f1c8c1: Status 404 returned error can't find the container with id ac90c756d832eb2c889cdeab38dd6d1a3d419d8aaff060fc5eb46ccf81f1c8c1 Jul 31 23:47:29.413970 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.411218 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wlkds"] Jul 31 23:47:29.420263 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:29.420234 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49904e8b_d888_442a_bab8_1e3c3f83d2e7.slice/crio-7246d336bef282157f9cedd0a7a3ea86968a203f6103f1ecf4acbaa16c07167f WatchSource:0}: Error finding container 7246d336bef282157f9cedd0a7a3ea86968a203f6103f1ecf4acbaa16c07167f: Status 404 returned error can't find the container with id 7246d336bef282157f9cedd0a7a3ea86968a203f6103f1ecf4acbaa16c07167f Jul 31 23:47:29.446675 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.446642 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dbgxq"] Jul 31 23:47:29.451220 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:29.451110 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9584b452_2eaa_4ff8_8ee3_ab4ed6fe4199.slice/crio-a142c2c6b26ac07c83d31f46e8a419ec74ca9b4de885b338c8b17d6f2076dfce WatchSource:0}: Error finding container a142c2c6b26ac07c83d31f46e8a419ec74ca9b4de885b338c8b17d6f2076dfce: Status 404 returned error can't find the container with id a142c2c6b26ac07c83d31f46e8a419ec74ca9b4de885b338c8b17d6f2076dfce Jul 31 23:47:29.636612 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.636487 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dbgxq" event={"ID":"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199","Type":"ContainerStarted","Data":"a142c2c6b26ac07c83d31f46e8a419ec74ca9b4de885b338c8b17d6f2076dfce"} Jul 31 23:47:29.638248 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.638182 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wlkds" event={"ID":"49904e8b-d888-442a-bab8-1e3c3f83d2e7","Type":"ContainerStarted","Data":"7246d336bef282157f9cedd0a7a3ea86968a203f6103f1ecf4acbaa16c07167f"} Jul 31 23:47:29.640022 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:29.639968 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l9sh5" event={"ID":"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee","Type":"ContainerStarted","Data":"ac90c756d832eb2c889cdeab38dd6d1a3d419d8aaff060fc5eb46ccf81f1c8c1"} Jul 31 23:47:31.473854 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:31.473806 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:31.477486 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:31.477402 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/053b510c-cca6-4383-84b2-836a9d686abc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5xm8p\" (UID: \"053b510c-cca6-4383-84b2-836a9d686abc\") " pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:31.710538 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:31.710502 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fst58\"" Jul 31 23:47:31.718171 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:31.717966 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5xm8p" Jul 31 23:47:37.891299 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.891270 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6gc4v"] Jul 31 23:47:37.903199 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.902638 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:37.905433 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.905343 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 31 23:47:37.906904 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.905931 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 31 23:47:37.906904 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.906124 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 31 23:47:37.906904 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.906285 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 31 23:47:37.906904 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.906492 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 31 23:47:37.906904 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.906507 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cmkpk\"" Jul 31 23:47:37.906904 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.906661 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 31 23:47:37.912275 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.912099 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9"] Jul 31 23:47:37.922578 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.922554 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:37.925736 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.925673 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 31 23:47:37.926556 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.926166 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gfgqv\"" Jul 31 23:47:37.926556 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.926363 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 31 23:47:37.939491 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.937528 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9"] Jul 31 23:47:37.941805 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.941779 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9"] Jul 31 23:47:37.950092 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.949964 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:37.953555 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.953077 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 31 23:47:37.953555 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.953161 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 31 23:47:37.953555 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.953077 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 31 23:47:37.953555 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.953393 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-thttx\"" Jul 31 23:47:37.961830 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:37.961664 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9"] Jul 31 23:47:38.029546 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029345 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61d9f4f9-7224-4b47-8e7c-e939af9bc010-metrics-client-ca\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.029546 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029396 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-accelerators-collector-config\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.029546 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029446 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-wtmp\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.029546 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029476 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfkjt\" (UniqueName: \"kubernetes.io/projected/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-api-access-nfkjt\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.029546 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029543 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.029915 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029582 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.029915 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029615 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.029915 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029651 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61d9f4f9-7224-4b47-8e7c-e939af9bc010-root\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.029915 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029677 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6kf6\" (UniqueName: \"kubernetes.io/projected/61d9f4f9-7224-4b47-8e7c-e939af9bc010-kube-api-access-q6kf6\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.029915 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029709 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eaf76d4-9339-44fc-9969-5b7e95509dbd-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.029915 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029730 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61d9f4f9-7224-4b47-8e7c-e939af9bc010-sys\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.029915 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029759 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eaf76d4-9339-44fc-9969-5b7e95509dbd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.029915 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029815 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.029915 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029844 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-textfile\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.029915 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029881 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.030344 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029920 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxqph\" (UniqueName: \"kubernetes.io/projected/7eaf76d4-9339-44fc-9969-5b7e95509dbd-kube-api-access-pxqph\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.030344 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029945 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-tls\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.030344 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.029977 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.030344 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.030007 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eaf76d4-9339-44fc-9969-5b7e95509dbd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130627 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-tls\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130689 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130723 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eaf76d4-9339-44fc-9969-5b7e95509dbd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130759 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61d9f4f9-7224-4b47-8e7c-e939af9bc010-metrics-client-ca\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-accelerators-collector-config\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130811 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-wtmp\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130836 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfkjt\" (UniqueName: \"kubernetes.io/projected/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-api-access-nfkjt\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130867 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130891 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130919 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130960 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61d9f4f9-7224-4b47-8e7c-e939af9bc010-root\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.130988 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6kf6\" (UniqueName: \"kubernetes.io/projected/61d9f4f9-7224-4b47-8e7c-e939af9bc010-kube-api-access-q6kf6\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.131017 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eaf76d4-9339-44fc-9969-5b7e95509dbd-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.131039 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61d9f4f9-7224-4b47-8e7c-e939af9bc010-sys\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.131069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eaf76d4-9339-44fc-9969-5b7e95509dbd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.132460 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.131106 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.131134 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-textfile\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.131171 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.131219 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxqph\" (UniqueName: \"kubernetes.io/projected/7eaf76d4-9339-44fc-9969-5b7e95509dbd-kube-api-access-pxqph\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.131397 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61d9f4f9-7224-4b47-8e7c-e939af9bc010-metrics-client-ca\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:38.131516 2582 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:38.131575 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eaf76d4-9339-44fc-9969-5b7e95509dbd-openshift-state-metrics-tls podName:7eaf76d4-9339-44fc-9969-5b7e95509dbd nodeName:}" failed. No retries permitted until 2026-07-31 23:47:38.631555115 +0000 UTC m=+75.791404208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7eaf76d4-9339-44fc-9969-5b7e95509dbd-openshift-state-metrics-tls") pod "openshift-state-metrics-6bb4d7b4dd-zq5c9" (UID: "7eaf76d4-9339-44fc-9969-5b7e95509dbd") : secret "openshift-state-metrics-tls" not found Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.132095 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-wtmp\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:38.132114 2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.132181 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-accelerators-collector-config\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.132263 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.132341 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61d9f4f9-7224-4b47-8e7c-e939af9bc010-root\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:38.132440 2582 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:38.132484 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-tls podName:61d9f4f9-7224-4b47-8e7c-e939af9bc010 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:38.632461778 +0000 UTC m=+75.792310886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-tls") pod "node-exporter-6gc4v" (UID: "61d9f4f9-7224-4b47-8e7c-e939af9bc010") : secret "node-exporter-tls" not found Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.132498 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61d9f4f9-7224-4b47-8e7c-e939af9bc010-sys\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:38.132506 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-tls podName:8c28b380-c1ce-4d77-9942-564a8ad8dbc5 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:38.632496352 +0000 UTC m=+75.792345448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-tls") pod "kube-state-metrics-6bd9664c4-rlmd9" (UID: "8c28b380-c1ce-4d77-9942-564a8ad8dbc5") : secret "kube-state-metrics-tls" not found Jul 31 23:47:38.133359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.132875 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eaf76d4-9339-44fc-9969-5b7e95509dbd-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.134168 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.133299 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-textfile\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.134168 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.134103 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.134517 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.134489 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.135509 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.135464 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eaf76d4-9339-44fc-9969-5b7e95509dbd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.136451 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.136395 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.141395 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.141335 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxqph\" (UniqueName: \"kubernetes.io/projected/7eaf76d4-9339-44fc-9969-5b7e95509dbd-kube-api-access-pxqph\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.142997 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.142976 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfkjt\" (UniqueName: \"kubernetes.io/projected/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-api-access-nfkjt\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.147171 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.147138 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6kf6\" (UniqueName: \"kubernetes.io/projected/61d9f4f9-7224-4b47-8e7c-e939af9bc010-kube-api-access-q6kf6\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.149627 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.149168 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.635855 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.635813 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.636023 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.635885 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-tls\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.636023 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.635923 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eaf76d4-9339-44fc-9969-5b7e95509dbd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.638637 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.638605 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61d9f4f9-7224-4b47-8e7c-e939af9bc010-node-exporter-tls\") pod \"node-exporter-6gc4v\" (UID: \"61d9f4f9-7224-4b47-8e7c-e939af9bc010\") " pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.638752 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.638658 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c28b380-c1ce-4d77-9942-564a8ad8dbc5-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-rlmd9\" (UID: \"8c28b380-c1ce-4d77-9942-564a8ad8dbc5\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:38.638830 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.638767 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eaf76d4-9339-44fc-9969-5b7e95509dbd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-zq5c9\" (UID: \"7eaf76d4-9339-44fc-9969-5b7e95509dbd\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.817536 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.817498 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6gc4v" Jul 31 23:47:38.835684 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.835651 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" Jul 31 23:47:38.863042 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:38.863007 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" Jul 31 23:47:42.658856 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.658797 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj"] Jul 31 23:47:42.673392 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.673374 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj"] Jul 31 23:47:42.673507 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.673474 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" Jul 31 23:47:42.676032 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.676010 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vkdbx\"" Jul 31 23:47:42.676151 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.676010 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 31 23:47:42.724942 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:42.724905 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61d9f4f9_7224_4b47_8e7c_e939af9bc010.slice/crio-01010bde17f117c05869f0076f155b57ec123939347ae5bf2ca57b0e7781a131 WatchSource:0}: Error finding container 01010bde17f117c05869f0076f155b57ec123939347ae5bf2ca57b0e7781a131: Status 404 returned error can't find the container with id 01010bde17f117c05869f0076f155b57ec123939347ae5bf2ca57b0e7781a131 Jul 31 23:47:42.772986 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.772803 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2cd92d2-0334-4923-b45d-35da209bdc80-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-xwmhj\" (UID: \"b2cd92d2-0334-4923-b45d-35da209bdc80\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" Jul 31 23:47:42.772986 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.772912 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7b7m\" (UniqueName: \"kubernetes.io/projected/b2cd92d2-0334-4923-b45d-35da209bdc80-kube-api-access-t7b7m\") pod \"monitoring-plugin-6dffbbd67-xwmhj\" (UID: \"b2cd92d2-0334-4923-b45d-35da209bdc80\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" Jul 31 23:47:42.874897 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.874055 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2cd92d2-0334-4923-b45d-35da209bdc80-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-xwmhj\" (UID: \"b2cd92d2-0334-4923-b45d-35da209bdc80\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" Jul 31 23:47:42.874897 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.874168 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7b7m\" (UniqueName: \"kubernetes.io/projected/b2cd92d2-0334-4923-b45d-35da209bdc80-kube-api-access-t7b7m\") pod \"monitoring-plugin-6dffbbd67-xwmhj\" (UID: \"b2cd92d2-0334-4923-b45d-35da209bdc80\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" Jul 31 23:47:42.874897 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:42.874645 2582 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 31 23:47:42.874897 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:47:42.874716 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2cd92d2-0334-4923-b45d-35da209bdc80-monitoring-plugin-cert podName:b2cd92d2-0334-4923-b45d-35da209bdc80 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:43.374691707 +0000 UTC m=+80.534541017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b2cd92d2-0334-4923-b45d-35da209bdc80-monitoring-plugin-cert") pod "monitoring-plugin-6dffbbd67-xwmhj" (UID: "b2cd92d2-0334-4923-b45d-35da209bdc80") : secret "monitoring-plugin-cert" not found Jul 31 23:47:42.890875 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.890788 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7b7m\" (UniqueName: \"kubernetes.io/projected/b2cd92d2-0334-4923-b45d-35da209bdc80-kube-api-access-t7b7m\") pod \"monitoring-plugin-6dffbbd67-xwmhj\" (UID: \"b2cd92d2-0334-4923-b45d-35da209bdc80\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" Jul 31 23:47:42.901093 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.898223 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5xm8p"] Jul 31 23:47:42.923200 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.921623 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6859f76475-9q2dc"] Jul 31 23:47:42.944382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.942348 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9"] Jul 31 23:47:42.944382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.944221 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c8ccc85ff-c96mc"] Jul 31 23:47:42.948825 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:42.948790 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c28b380_c1ce_4d77_9942_564a8ad8dbc5.slice/crio-ea3103b43ebe2971188d69643e32c52deee4c79713fd104774e59eec2011e609 WatchSource:0}: Error finding container ea3103b43ebe2971188d69643e32c52deee4c79713fd104774e59eec2011e609: Status 404 returned error can't find the container with id ea3103b43ebe2971188d69643e32c52deee4c79713fd104774e59eec2011e609 Jul 31 23:47:42.966158 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.964112 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:42.980903 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.978873 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c8ccc85ff-c96mc"] Jul 31 23:47:42.985385 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.985359 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9"] Jul 31 23:47:42.987100 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:42.986887 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 31 23:47:43.084975 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.084940 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-oauth-config\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.085180 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.085161 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv4n7\" (UniqueName: \"kubernetes.io/projected/3f680908-4540-4d6d-baef-9e2142a32e3a-kube-api-access-rv4n7\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.085301 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.085287 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-oauth-serving-cert\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.085446 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.085430 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-service-ca\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.085545 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.085532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-serving-cert\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.085697 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.085681 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-console-config\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.085881 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.085866 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-trusted-ca-bundle\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.188481 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.188444 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-serving-cert\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.188638 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.188514 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-console-config\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.188638 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.188553 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-trusted-ca-bundle\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.188638 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.188592 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-oauth-config\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.188638 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.188627 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv4n7\" (UniqueName: \"kubernetes.io/projected/3f680908-4540-4d6d-baef-9e2142a32e3a-kube-api-access-rv4n7\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.188842 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.188652 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-oauth-serving-cert\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.188842 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.188704 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-service-ca\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.191482 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.190118 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-service-ca\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.192199 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.192178 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-console-config\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.192363 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.192336 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-oauth-serving-cert\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.192860 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.192809 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-trusted-ca-bundle\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.199887 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.199694 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-serving-cert\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.200506 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.200462 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-oauth-config\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.206400 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.206308 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv4n7\" (UniqueName: \"kubernetes.io/projected/3f680908-4540-4d6d-baef-9e2142a32e3a-kube-api-access-rv4n7\") pod \"console-7c8ccc85ff-c96mc\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.300299 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.300180 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:43.394928 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.394819 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2cd92d2-0334-4923-b45d-35da209bdc80-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-xwmhj\" (UID: \"b2cd92d2-0334-4923-b45d-35da209bdc80\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" Jul 31 23:47:43.397946 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.397891 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2cd92d2-0334-4923-b45d-35da209bdc80-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-xwmhj\" (UID: \"b2cd92d2-0334-4923-b45d-35da209bdc80\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" Jul 31 23:47:43.483974 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.483895 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c8ccc85ff-c96mc"] Jul 31 23:47:43.485298 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:43.485256 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f680908_4540_4d6d_baef_9e2142a32e3a.slice/crio-65294873f1daf378a465e4414fbeeb80647a67a5d1a26cb448eee6b6de8b4127 WatchSource:0}: Error finding container 65294873f1daf378a465e4414fbeeb80647a67a5d1a26cb448eee6b6de8b4127: Status 404 returned error can't find the container with id 65294873f1daf378a465e4414fbeeb80647a67a5d1a26cb448eee6b6de8b4127 Jul 31 23:47:43.585613 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.585326 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" Jul 31 23:47:43.708428 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.705353 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l9sh5" event={"ID":"03fa2c21-f2f9-450a-ac02-5a0de6eef3ee","Type":"ContainerStarted","Data":"5d6a32bd473c52ca1f94ddeb5380eca95c19322395147b1bb919f3206b95a40c"} Jul 31 23:47:43.708428 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.707639 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" event={"ID":"8c28b380-c1ce-4d77-9942-564a8ad8dbc5","Type":"ContainerStarted","Data":"ea3103b43ebe2971188d69643e32c52deee4c79713fd104774e59eec2011e609"} Jul 31 23:47:43.715721 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.713571 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-6qbq8" event={"ID":"8b00e295-a506-43f7-b005-3de7ed37ea94","Type":"ContainerStarted","Data":"0489b772135d1c8e9564348c21828ba9fedf1b065cb2b1d8022c5f75e15e5088"} Jul 31 23:47:43.715880 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.715771 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-6qbq8" Jul 31 23:47:43.718551 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.718313 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-6qbq8" Jul 31 23:47:43.720204 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.719776 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6859f76475-9q2dc" event={"ID":"b27773f2-7e52-4c78-8bac-1c23a9629369","Type":"ContainerStarted","Data":"e5aa2cb69de1e63092bda718f6fa66d027004c20a936774b0f899706c3e9e339"} Jul 31 23:47:43.735349 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.735249 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dbgxq" event={"ID":"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199","Type":"ContainerStarted","Data":"421d79769d89b61169644da2fa6f9c66f03a26e2470a3190ad581640d9a6a110"} Jul 31 23:47:43.735349 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.735306 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dbgxq" event={"ID":"9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199","Type":"ContainerStarted","Data":"d0e27db062a9cc20f20219d8dd67622a488e188755145ae61773266aa02853e3"} Jul 31 23:47:43.740426 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.739962 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wlkds" event={"ID":"49904e8b-d888-442a-bab8-1e3c3f83d2e7","Type":"ContainerStarted","Data":"81b38a8805a5f9543dd8c4915037969d576c65e0c993205f3cce6d0355332847"} Jul 31 23:47:43.740629 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.740575 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:47:43.752142 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.748550 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-l9sh5" podStartSLOduration=67.406762605 podStartE2EDuration="1m20.748529899s" totalImagesPullingTime="13.341767294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:43.725052823 +0000 UTC m=+80.884901936" watchObservedRunningTime="2026-07-31 23:47:43.748529899 +0000 UTC m=+80.908379014" Jul 31 23:47:43.758721 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.758017 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8ccc85ff-c96mc" event={"ID":"3f680908-4540-4d6d-baef-9e2142a32e3a","Type":"ContainerStarted","Data":"b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f"} Jul 31 23:47:43.758721 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.758069 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8ccc85ff-c96mc" event={"ID":"3f680908-4540-4d6d-baef-9e2142a32e3a","Type":"ContainerStarted","Data":"65294873f1daf378a465e4414fbeeb80647a67a5d1a26cb448eee6b6de8b4127"} Jul 31 23:47:43.761890 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.761755 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" event={"ID":"7eaf76d4-9339-44fc-9969-5b7e95509dbd","Type":"ContainerStarted","Data":"da650e15cb13084797fc4cdadb384613119b9f681452ae876bd9ab1b37b96363"} Jul 31 23:47:43.761890 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.761814 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" event={"ID":"7eaf76d4-9339-44fc-9969-5b7e95509dbd","Type":"ContainerStarted","Data":"4bdd74936a8e29f7ed880c0557be30c4f153614807a41f62437a39ff1bb05048"} Jul 31 23:47:43.761890 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.761843 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" event={"ID":"7eaf76d4-9339-44fc-9969-5b7e95509dbd","Type":"ContainerStarted","Data":"cc759e0eeaf19dd2200b7c0ae3d7712e45a4c27738b7a1f90eb831d95c286200"} Jul 31 23:47:43.766512 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.766457 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6gc4v" event={"ID":"61d9f4f9-7224-4b47-8e7c-e939af9bc010","Type":"ContainerStarted","Data":"01010bde17f117c05869f0076f155b57ec123939347ae5bf2ca57b0e7781a131"} Jul 31 23:47:43.769161 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.769116 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-6qbq8" podStartSLOduration=1.9883662850000001 podStartE2EDuration="21.769097307s" totalImagesPullingTime="19.780731022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:43.767353051 +0000 UTC m=+80.927202164" watchObservedRunningTime="2026-07-31 23:47:43.769097307 +0000 UTC m=+80.928946419" Jul 31 23:47:43.769263 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.769211 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6859f76475-9q2dc" podStartSLOduration=2.1742151180000002 podStartE2EDuration="16.769204464s" totalImagesPullingTime="14.594989346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:43.749055919 +0000 UTC m=+80.908905034" watchObservedRunningTime="2026-07-31 23:47:43.769204464 +0000 UTC m=+80.929053576" Jul 31 23:47:43.779884 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.779853 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5xm8p" event={"ID":"053b510c-cca6-4383-84b2-836a9d686abc","Type":"ContainerStarted","Data":"d2b687b280a9b2a9f358e91557124663b8ed960694d92d6db9e790ccb114006d"} Jul 31 23:47:43.780036 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.779914 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5xm8p" event={"ID":"053b510c-cca6-4383-84b2-836a9d686abc","Type":"ContainerStarted","Data":"54bcd37eea96df403a3b16c7a8b4285009ff2f36342cac3742a34ac1204ff9e4"} Jul 31 23:47:43.793646 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.793532 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wlkds" podStartSLOduration=67.496235865 podStartE2EDuration="1m20.793511311s" totalImagesPullingTime="13.297275446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:43.792061508 +0000 UTC m=+80.951910622" watchObservedRunningTime="2026-07-31 23:47:43.793511311 +0000 UTC m=+80.953360403" Jul 31 23:47:43.807890 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.807857 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj"] Jul 31 23:47:43.809388 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.809287 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dbgxq" podStartSLOduration=67.549793764 podStartE2EDuration="1m20.809268061s" totalImagesPullingTime="13.259474297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:43.807131731 +0000 UTC m=+80.966980844" watchObservedRunningTime="2026-07-31 23:47:43.809268061 +0000 UTC m=+80.969117178" Jul 31 23:47:43.832482 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:43.832308 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c8ccc85ff-c96mc" podStartSLOduration=1.8322845779999999 podStartE2EDuration="1.832284578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:43.830819596 +0000 UTC m=+80.990668711" watchObservedRunningTime="2026-07-31 23:47:43.832284578 +0000 UTC m=+80.992133691" Jul 31 23:47:44.125349 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.123074 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 23:47:44.141575 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.141534 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.145985 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.145456 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 31 23:47:44.145985 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.145477 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6fqa7uss2arjn\"" Jul 31 23:47:44.145985 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.145696 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bpcdg\"" Jul 31 23:47:44.145985 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.145711 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 31 23:47:44.145985 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.145832 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 31 23:47:44.146362 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.146123 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 31 23:47:44.146362 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.146143 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 31 23:47:44.146362 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.146231 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 31 23:47:44.146362 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.146314 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 31 23:47:44.146619 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.146429 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 31 23:47:44.146619 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.146443 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 31 23:47:44.146735 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.146686 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 31 23:47:44.148833 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.146787 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 31 23:47:44.148833 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.146867 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 31 23:47:44.148833 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.148147 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 31 23:47:44.154972 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.149776 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 31 23:47:44.157859 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.156679 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 31 23:47:44.158790 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.158767 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204198 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/669a0ac6-317a-46c8-89e8-56c0291a9b7f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204273 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204309 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204335 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204359 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204435 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204465 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-config\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204489 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204522 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204545 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204570 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204593 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.204650 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204616 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-web-config\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.205485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204660 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cv2c\" (UniqueName: \"kubernetes.io/projected/669a0ac6-317a-46c8-89e8-56c0291a9b7f-kube-api-access-6cv2c\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.205485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204727 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.205485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204778 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.205485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204813 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.205485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204859 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.205485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204920 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/669a0ac6-317a-46c8-89e8-56c0291a9b7f-config-out\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.205485 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.204958 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.306207 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306163 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/669a0ac6-317a-46c8-89e8-56c0291a9b7f-config-out\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.306207 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306213 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306259 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/669a0ac6-317a-46c8-89e8-56c0291a9b7f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306284 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306313 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306339 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306366 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306425 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306455 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-config\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306479 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306514 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306533 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306557 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306583 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-web-config\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306631 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cv2c\" (UniqueName: \"kubernetes.io/projected/669a0ac6-317a-46c8-89e8-56c0291a9b7f-kube-api-access-6cv2c\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306672 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306723 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.307862 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306750 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.308707 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.306781 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.308707 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.307589 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.308707 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.308677 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.309822 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.309500 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.310672 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.310648 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.310989 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.310962 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.313461 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.312727 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-web-config\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.313461 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.312941 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/669a0ac6-317a-46c8-89e8-56c0291a9b7f-config-out\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.313623 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.313391 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.313705 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.313660 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.314366 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.314337 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.314565 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.314351 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/669a0ac6-317a-46c8-89e8-56c0291a9b7f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.318246 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.318211 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-config\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.320014 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.318694 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.320014 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.319052 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.320014 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.319959 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.320399 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.320363 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.321149 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.321122 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/669a0ac6-317a-46c8-89e8-56c0291a9b7f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.321847 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.321808 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.322389 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.322367 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/669a0ac6-317a-46c8-89e8-56c0291a9b7f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.324993 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.324976 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cv2c\" (UniqueName: \"kubernetes.io/projected/669a0ac6-317a-46c8-89e8-56c0291a9b7f-kube-api-access-6cv2c\") pod \"prometheus-k8s-0\" (UID: \"669a0ac6-317a-46c8-89e8-56c0291a9b7f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.458420 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.458306 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:47:44.523668 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:44.523631 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2cd92d2_0334_4923_b45d_35da209bdc80.slice/crio-dae6c27c38fb1dbdf4be2480b6d1f50302e3bc2e6cef470afe7e647d6cbf4edb WatchSource:0}: Error finding container dae6c27c38fb1dbdf4be2480b6d1f50302e3bc2e6cef470afe7e647d6cbf4edb: Status 404 returned error can't find the container with id dae6c27c38fb1dbdf4be2480b6d1f50302e3bc2e6cef470afe7e647d6cbf4edb Jul 31 23:47:44.614523 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.614439 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-659bffd457-92tkj" Jul 31 23:47:44.786713 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:44.786150 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" event={"ID":"b2cd92d2-0334-4923-b45d-35da209bdc80","Type":"ContainerStarted","Data":"dae6c27c38fb1dbdf4be2480b6d1f50302e3bc2e6cef470afe7e647d6cbf4edb"} Jul 31 23:47:46.019746 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:46.019642 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 31 23:47:46.027207 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:47:46.026631 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod669a0ac6_317a_46c8_89e8_56c0291a9b7f.slice/crio-6b1904015c478f5964093567b634d1719c8e77011f03a99eb42574d7ffdb770d WatchSource:0}: Error finding container 6b1904015c478f5964093567b634d1719c8e77011f03a99eb42574d7ffdb770d: Status 404 returned error can't find the container with id 6b1904015c478f5964093567b634d1719c8e77011f03a99eb42574d7ffdb770d Jul 31 23:47:46.800422 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:46.800326 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" event={"ID":"7eaf76d4-9339-44fc-9969-5b7e95509dbd","Type":"ContainerStarted","Data":"1ac20a6382d0b6eb63b1be9eb463eebb0666859dfaf8c6b96561c4b260a6b000"} Jul 31 23:47:46.802904 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:46.802864 2582 generic.go:356] "Generic (PLEG): container finished" podID="61d9f4f9-7224-4b47-8e7c-e939af9bc010" containerID="5cbec0ee39d9c4116ecdef6c403c6186e51979efbda3384d0b96bc8d1b2f1656" exitCode=0 Jul 31 23:47:46.803062 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:46.802955 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6gc4v" event={"ID":"61d9f4f9-7224-4b47-8e7c-e939af9bc010","Type":"ContainerDied","Data":"5cbec0ee39d9c4116ecdef6c403c6186e51979efbda3384d0b96bc8d1b2f1656"} Jul 31 23:47:46.806744 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:46.806709 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5xm8p" event={"ID":"053b510c-cca6-4383-84b2-836a9d686abc","Type":"ContainerStarted","Data":"abc5181826713b99f46433b91cad9265fb23e287e9dfb59486881f08818a2f2a"} Jul 31 23:47:46.811766 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:46.811739 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"669a0ac6-317a-46c8-89e8-56c0291a9b7f","Type":"ContainerStarted","Data":"6b1904015c478f5964093567b634d1719c8e77011f03a99eb42574d7ffdb770d"} Jul 31 23:47:46.817524 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:46.817232 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" event={"ID":"8c28b380-c1ce-4d77-9942-564a8ad8dbc5","Type":"ContainerStarted","Data":"b71b3314c56d8d54d463218ff9050511a5f3c34596f39e8e0b6fb80b1899d2b3"} Jul 31 23:47:46.817524 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:46.817282 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" event={"ID":"8c28b380-c1ce-4d77-9942-564a8ad8dbc5","Type":"ContainerStarted","Data":"f9c7b9f48ee6e7e9c32d0720dc7c63e63deefe9d41e1d7c422a03badfb16b189"} Jul 31 23:47:46.817524 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:46.817310 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" event={"ID":"8c28b380-c1ce-4d77-9942-564a8ad8dbc5","Type":"ContainerStarted","Data":"9a3ec3d3e2eb0bec3d65c75e397412a73053b5de3638251b31e741ecd92f81fb"} Jul 31 23:47:46.835468 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:46.835424 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zq5c9" podStartSLOduration=7.205604049 podStartE2EDuration="9.835389335s" totalImagesPullingTime="2.629785286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:46.832940516 +0000 UTC m=+83.992789631" watchObservedRunningTime="2026-07-31 23:47:46.835389335 +0000 UTC m=+83.995238449" Jul 31 23:47:46.886927 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:46.886858 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-rlmd9" podStartSLOduration=7.011791547 podStartE2EDuration="9.886839398s" totalImagesPullingTime="2.875047851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:46.886047478 +0000 UTC m=+84.045896591" watchObservedRunningTime="2026-07-31 23:47:46.886839398 +0000 UTC m=+84.046688512" Jul 31 23:47:47.929838 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:47.929790 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:47:48.829467 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:48.829429 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6gc4v" event={"ID":"61d9f4f9-7224-4b47-8e7c-e939af9bc010","Type":"ContainerStarted","Data":"787e279a3658a3f644c6977d10eeb5d1b621b0c3c21076451322eb2e0b2c7d39"} Jul 31 23:47:48.835725 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:48.835008 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"669a0ac6-317a-46c8-89e8-56c0291a9b7f","Type":"ContainerStarted","Data":"2849014ddc6447271dded5361ff9ee8ce82204ce3eef0efa97a51fd87f119cbd"} Jul 31 23:47:48.837913 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:48.837881 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" event={"ID":"b2cd92d2-0334-4923-b45d-35da209bdc80","Type":"ContainerStarted","Data":"327a176cd1ad68f96ac6a0b8b2e7ef5d78213cac40c41728e7d4a11b8bb090f8"} Jul 31 23:47:48.838125 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:48.838106 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" Jul 31 23:47:48.844236 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:48.844206 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" Jul 31 23:47:48.876278 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:48.876226 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-xwmhj" podStartSLOduration=2.969299838 podStartE2EDuration="6.876199073s" totalImagesPullingTime="3.906899235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:48.875818773 +0000 UTC m=+86.035667898" watchObservedRunningTime="2026-07-31 23:47:48.876199073 +0000 UTC m=+86.036048192" Jul 31 23:47:49.844552 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:49.844510 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6gc4v" event={"ID":"61d9f4f9-7224-4b47-8e7c-e939af9bc010","Type":"ContainerStarted","Data":"98b6de80159ed31c8ef7e70f4db8b0a56b47638670c5953d3009600885344d74"} Jul 31 23:47:49.847136 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:49.847100 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5xm8p" event={"ID":"053b510c-cca6-4383-84b2-836a9d686abc","Type":"ContainerStarted","Data":"1ea0cfa2201e2dc50d2536c5663aa3d6bfe2be66ecff74e5996f3698afc4f00e"} Jul 31 23:47:49.848940 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:49.848913 2582 generic.go:356] "Generic (PLEG): container finished" podID="669a0ac6-317a-46c8-89e8-56c0291a9b7f" containerID="2849014ddc6447271dded5361ff9ee8ce82204ce3eef0efa97a51fd87f119cbd" exitCode=0 Jul 31 23:47:49.849069 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:49.848995 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"669a0ac6-317a-46c8-89e8-56c0291a9b7f","Type":"ContainerDied","Data":"2849014ddc6447271dded5361ff9ee8ce82204ce3eef0efa97a51fd87f119cbd"} Jul 31 23:47:49.863613 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:49.863556 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6gc4v" podStartSLOduration=9.595926806 podStartE2EDuration="12.863537707s" totalImagesPullingTime="3.09937133s" totalInitContainerRuntime="168.239571ms" isStatefulPod=true podCreationTimestamp="2026-07-31 23:47:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:49.863296267 +0000 UTC m=+87.023145378" watchObservedRunningTime="2026-07-31 23:47:49.863537707 +0000 UTC m=+87.023386819" Jul 31 23:47:49.882583 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:49.882531 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5xm8p" podStartSLOduration=45.291869782 podStartE2EDuration="50.882519264s" totalImagesPullingTime="5.590649482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:49.882120383 +0000 UTC m=+87.041969494" watchObservedRunningTime="2026-07-31 23:47:49.882519264 +0000 UTC m=+87.042368376" Jul 31 23:47:53.300455 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:53.300392 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:53.301056 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:53.300483 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:53.307396 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:53.307367 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:53.871370 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:53.871289 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"669a0ac6-317a-46c8-89e8-56c0291a9b7f","Type":"ContainerStarted","Data":"3828d8acb654f6315afbc3277bcb6cb4e5d2361dc031807e4400fa5105efa25e"} Jul 31 23:47:53.871569 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:53.871386 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"669a0ac6-317a-46c8-89e8-56c0291a9b7f","Type":"ContainerStarted","Data":"b108b4845447df4aff20651f1e49c2dc69bf07dc7bea92a5853d1257d7af0002"} Jul 31 23:47:53.876450 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:53.876430 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:47:56.886298 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:56.886263 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"669a0ac6-317a-46c8-89e8-56c0291a9b7f","Type":"ContainerStarted","Data":"efd5e11bb89cfdeec5273d8c13e3ea318a488dda84529d5b5e23e0c83c16236f"} Jul 31 23:47:56.886729 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:56.886334 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"669a0ac6-317a-46c8-89e8-56c0291a9b7f","Type":"ContainerStarted","Data":"4e67f63d0e12194598d6954202a6fde82be71021cf0bf20909f1d5ca31da755f"} Jul 31 23:47:56.886729 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:56.886362 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"669a0ac6-317a-46c8-89e8-56c0291a9b7f","Type":"ContainerStarted","Data":"fc4f6b02e61d2de19efd099f150f0b4a439422c09ff5289dcd19852c6380f59e"} Jul 31 23:47:56.886729 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:56.886395 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"669a0ac6-317a-46c8-89e8-56c0291a9b7f","Type":"ContainerStarted","Data":"fe4969fd4869864ff88d5f64d21f8bc548967932a243b1be5e203c7157a6a16b"} Jul 31 23:47:56.915325 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:56.915274 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.672670443 podStartE2EDuration="12.915255297s" totalImagesPullingTime="8.22155458s" totalInitContainerRuntime="21.030274ms" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:56.912919661 +0000 UTC m=+94.072768770" watchObservedRunningTime="2026-07-31 23:47:56.915255297 +0000 UTC m=+94.075104409" Jul 31 23:47:59.458608 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:47:59.458573 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:48:08.772963 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:08.772884 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6859f76475-9q2dc" podUID="b27773f2-7e52-4c78-8bac-1c23a9629369" containerName="console" containerID="cri-o://e5aa2cb69de1e63092bda718f6fa66d027004c20a936774b0f899706c3e9e339" gracePeriod=15 Jul 31 23:48:08.937096 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:08.937066 2582 generic.go:356] "Generic (PLEG): container finished" podID="b27773f2-7e52-4c78-8bac-1c23a9629369" containerID="e5aa2cb69de1e63092bda718f6fa66d027004c20a936774b0f899706c3e9e339" exitCode=0 Jul 31 23:48:08.937258 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:08.937145 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6859f76475-9q2dc" event={"ID":"b27773f2-7e52-4c78-8bac-1c23a9629369","Type":"ContainerDied","Data":"e5aa2cb69de1e63092bda718f6fa66d027004c20a936774b0f899706c3e9e339"} Jul 31 23:48:09.052910 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.052876 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:48:09.125530 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.125499 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnd6n\" (UniqueName: \"kubernetes.io/projected/b27773f2-7e52-4c78-8bac-1c23a9629369-kube-api-access-nnd6n\") pod \"b27773f2-7e52-4c78-8bac-1c23a9629369\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " Jul 31 23:48:09.125712 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.125543 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-console-config\") pod \"b27773f2-7e52-4c78-8bac-1c23a9629369\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " Jul 31 23:48:09.125712 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.125571 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-oauth-config\") pod \"b27773f2-7e52-4c78-8bac-1c23a9629369\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " Jul 31 23:48:09.125712 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.125594 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-service-ca\") pod \"b27773f2-7e52-4c78-8bac-1c23a9629369\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " Jul 31 23:48:09.125712 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.125610 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-serving-cert\") pod \"b27773f2-7e52-4c78-8bac-1c23a9629369\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " Jul 31 23:48:09.125712 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.125641 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-oauth-serving-cert\") pod \"b27773f2-7e52-4c78-8bac-1c23a9629369\" (UID: \"b27773f2-7e52-4c78-8bac-1c23a9629369\") " Jul 31 23:48:09.126065 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.126037 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-console-config" (OuterVolumeSpecName: "console-config") pod "b27773f2-7e52-4c78-8bac-1c23a9629369" (UID: "b27773f2-7e52-4c78-8bac-1c23a9629369"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:48:09.126174 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.126147 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b27773f2-7e52-4c78-8bac-1c23a9629369" (UID: "b27773f2-7e52-4c78-8bac-1c23a9629369"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:48:09.126174 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.126166 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-service-ca" (OuterVolumeSpecName: "service-ca") pod "b27773f2-7e52-4c78-8bac-1c23a9629369" (UID: "b27773f2-7e52-4c78-8bac-1c23a9629369"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:48:09.127952 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.127933 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b27773f2-7e52-4c78-8bac-1c23a9629369-kube-api-access-nnd6n" (OuterVolumeSpecName: "kube-api-access-nnd6n") pod "b27773f2-7e52-4c78-8bac-1c23a9629369" (UID: "b27773f2-7e52-4c78-8bac-1c23a9629369"). InnerVolumeSpecName "kube-api-access-nnd6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:48:09.128214 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.128194 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b27773f2-7e52-4c78-8bac-1c23a9629369" (UID: "b27773f2-7e52-4c78-8bac-1c23a9629369"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 23:48:09.128305 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.128278 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b27773f2-7e52-4c78-8bac-1c23a9629369" (UID: "b27773f2-7e52-4c78-8bac-1c23a9629369"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 23:48:09.227198 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.227167 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-oauth-config\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:48:09.227198 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.227191 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-service-ca\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:48:09.227198 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.227203 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b27773f2-7e52-4c78-8bac-1c23a9629369-console-serving-cert\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:48:09.227429 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.227211 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-oauth-serving-cert\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:48:09.227429 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.227219 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnd6n\" (UniqueName: \"kubernetes.io/projected/b27773f2-7e52-4c78-8bac-1c23a9629369-kube-api-access-nnd6n\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:48:09.227429 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.227228 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b27773f2-7e52-4c78-8bac-1c23a9629369-console-config\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:48:09.942627 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.942597 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6859f76475-9q2dc" Jul 31 23:48:09.943036 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.942626 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6859f76475-9q2dc" event={"ID":"b27773f2-7e52-4c78-8bac-1c23a9629369","Type":"ContainerDied","Data":"73a7665f4d1262f587e2cb314da217abe0295d416185ab38a0791bc80c02e5f6"} Jul 31 23:48:09.943036 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.942683 2582 scope.go:117] "RemoveContainer" containerID="e5aa2cb69de1e63092bda718f6fa66d027004c20a936774b0f899706c3e9e339" Jul 31 23:48:09.960210 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.960185 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6859f76475-9q2dc"] Jul 31 23:48:09.965964 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:09.965943 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6859f76475-9q2dc"] Jul 31 23:48:11.370876 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:11.370838 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b27773f2-7e52-4c78-8bac-1c23a9629369" path="/var/lib/kubelet/pods/b27773f2-7e52-4c78-8bac-1c23a9629369/volumes" Jul 31 23:48:15.793745 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:15.793698 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wlkds" Jul 31 23:48:44.458737 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:44.458697 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:48:44.477312 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:44.477283 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:48:45.089829 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:48:45.089804 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 31 23:49:15.165160 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:15.165126 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c8ccc85ff-c96mc"] Jul 31 23:49:40.185210 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.185148 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c8ccc85ff-c96mc" podUID="3f680908-4540-4d6d-baef-9e2142a32e3a" containerName="console" containerID="cri-o://b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f" gracePeriod=15 Jul 31 23:49:40.440434 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.440369 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:49:40.588143 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.588106 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rv4n7\" (UniqueName: \"kubernetes.io/projected/3f680908-4540-4d6d-baef-9e2142a32e3a-kube-api-access-rv4n7\") pod \"3f680908-4540-4d6d-baef-9e2142a32e3a\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " Jul 31 23:49:40.588333 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.588174 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-service-ca\") pod \"3f680908-4540-4d6d-baef-9e2142a32e3a\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " Jul 31 23:49:40.588333 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.588205 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-oauth-serving-cert\") pod \"3f680908-4540-4d6d-baef-9e2142a32e3a\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " Jul 31 23:49:40.588333 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.588228 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-oauth-config\") pod \"3f680908-4540-4d6d-baef-9e2142a32e3a\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " Jul 31 23:49:40.588333 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.588246 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-console-config\") pod \"3f680908-4540-4d6d-baef-9e2142a32e3a\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " Jul 31 23:49:40.588333 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.588269 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-trusted-ca-bundle\") pod \"3f680908-4540-4d6d-baef-9e2142a32e3a\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " Jul 31 23:49:40.588333 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.588318 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-serving-cert\") pod \"3f680908-4540-4d6d-baef-9e2142a32e3a\" (UID: \"3f680908-4540-4d6d-baef-9e2142a32e3a\") " Jul 31 23:49:40.588758 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.588652 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-service-ca" (OuterVolumeSpecName: "service-ca") pod "3f680908-4540-4d6d-baef-9e2142a32e3a" (UID: "3f680908-4540-4d6d-baef-9e2142a32e3a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:49:40.588811 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.588781 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3f680908-4540-4d6d-baef-9e2142a32e3a" (UID: "3f680908-4540-4d6d-baef-9e2142a32e3a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:49:40.588894 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.588871 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3f680908-4540-4d6d-baef-9e2142a32e3a" (UID: "3f680908-4540-4d6d-baef-9e2142a32e3a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:49:40.588929 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.588917 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-console-config" (OuterVolumeSpecName: "console-config") pod "3f680908-4540-4d6d-baef-9e2142a32e3a" (UID: "3f680908-4540-4d6d-baef-9e2142a32e3a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:49:40.590306 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.590284 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f680908-4540-4d6d-baef-9e2142a32e3a-kube-api-access-rv4n7" (OuterVolumeSpecName: "kube-api-access-rv4n7") pod "3f680908-4540-4d6d-baef-9e2142a32e3a" (UID: "3f680908-4540-4d6d-baef-9e2142a32e3a"). InnerVolumeSpecName "kube-api-access-rv4n7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:49:40.590791 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.590759 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3f680908-4540-4d6d-baef-9e2142a32e3a" (UID: "3f680908-4540-4d6d-baef-9e2142a32e3a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 23:49:40.590791 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.590784 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3f680908-4540-4d6d-baef-9e2142a32e3a" (UID: "3f680908-4540-4d6d-baef-9e2142a32e3a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 23:49:40.689477 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.689442 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-serving-cert\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:49:40.689477 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.689470 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rv4n7\" (UniqueName: \"kubernetes.io/projected/3f680908-4540-4d6d-baef-9e2142a32e3a-kube-api-access-rv4n7\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:49:40.689477 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.689482 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-service-ca\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:49:40.689711 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.689493 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-oauth-serving-cert\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:49:40.689711 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.689502 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f680908-4540-4d6d-baef-9e2142a32e3a-console-oauth-config\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:49:40.689711 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.689510 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-console-config\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:49:40.689711 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:40.689520 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f680908-4540-4d6d-baef-9e2142a32e3a-trusted-ca-bundle\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:49:41.288306 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:41.288272 2582 generic.go:356] "Generic (PLEG): container finished" podID="3f680908-4540-4d6d-baef-9e2142a32e3a" containerID="b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f" exitCode=0 Jul 31 23:49:41.288768 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:41.288328 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8ccc85ff-c96mc" Jul 31 23:49:41.288768 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:41.288333 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8ccc85ff-c96mc" event={"ID":"3f680908-4540-4d6d-baef-9e2142a32e3a","Type":"ContainerDied","Data":"b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f"} Jul 31 23:49:41.288768 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:41.288394 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8ccc85ff-c96mc" event={"ID":"3f680908-4540-4d6d-baef-9e2142a32e3a","Type":"ContainerDied","Data":"65294873f1daf378a465e4414fbeeb80647a67a5d1a26cb448eee6b6de8b4127"} Jul 31 23:49:41.288768 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:41.288438 2582 scope.go:117] "RemoveContainer" containerID="b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f" Jul 31 23:49:41.296817 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:41.296798 2582 scope.go:117] "RemoveContainer" containerID="b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f" Jul 31 23:49:41.297076 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:49:41.297054 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f\": container with ID starting with b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f not found: ID does not exist" containerID="b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f" Jul 31 23:49:41.297119 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:41.297085 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f"} err="failed to get container status \"b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f\": rpc error: code = NotFound desc = could not find container \"b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f\": container with ID starting with b349b8501893b80e42f30f57ff15cdb636806f9be661b5068e2d125e9dff445f not found: ID does not exist" Jul 31 23:49:41.308985 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:41.308951 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c8ccc85ff-c96mc"] Jul 31 23:49:41.312147 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:41.312123 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c8ccc85ff-c96mc"] Jul 31 23:49:41.372594 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:49:41.372563 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f680908-4540-4d6d-baef-9e2142a32e3a" path="/var/lib/kubelet/pods/3f680908-4540-4d6d-baef-9e2142a32e3a/volumes" Jul 31 23:51:23.256281 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:51:23.256257 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 31 23:53:45.540902 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.540825 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-qhnrw"] Jul 31 23:53:45.541341 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.541124 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f680908-4540-4d6d-baef-9e2142a32e3a" containerName="console" Jul 31 23:53:45.541341 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.541137 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f680908-4540-4d6d-baef-9e2142a32e3a" containerName="console" Jul 31 23:53:45.541341 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.541149 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b27773f2-7e52-4c78-8bac-1c23a9629369" containerName="console" Jul 31 23:53:45.541341 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.541155 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b27773f2-7e52-4c78-8bac-1c23a9629369" containerName="console" Jul 31 23:53:45.541341 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.541201 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f680908-4540-4d6d-baef-9e2142a32e3a" containerName="console" Jul 31 23:53:45.541341 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.541210 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b27773f2-7e52-4c78-8bac-1c23a9629369" containerName="console" Jul 31 23:53:45.543247 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.543230 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qhnrw" Jul 31 23:53:45.545819 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.545795 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-5dcjg\"" Jul 31 23:53:45.545957 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.545845 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 31 23:53:45.545957 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.545905 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 31 23:53:45.546861 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.546847 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 31 23:53:45.550701 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.550681 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-qhnrw"] Jul 31 23:53:45.654048 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.654015 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0277ebea-21b6-4021-b285-cee18ac86e28-model-cache\") pod \"s3-init-qhnrw\" (UID: \"0277ebea-21b6-4021-b285-cee18ac86e28\") " pod="kserve/s3-init-qhnrw" Jul 31 23:53:45.654219 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.654066 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xkkx\" (UniqueName: \"kubernetes.io/projected/0277ebea-21b6-4021-b285-cee18ac86e28-kube-api-access-9xkkx\") pod \"s3-init-qhnrw\" (UID: \"0277ebea-21b6-4021-b285-cee18ac86e28\") " pod="kserve/s3-init-qhnrw" Jul 31 23:53:45.754498 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.754462 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xkkx\" (UniqueName: \"kubernetes.io/projected/0277ebea-21b6-4021-b285-cee18ac86e28-kube-api-access-9xkkx\") pod \"s3-init-qhnrw\" (UID: \"0277ebea-21b6-4021-b285-cee18ac86e28\") " pod="kserve/s3-init-qhnrw" Jul 31 23:53:45.754635 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.754614 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0277ebea-21b6-4021-b285-cee18ac86e28-model-cache\") pod \"s3-init-qhnrw\" (UID: \"0277ebea-21b6-4021-b285-cee18ac86e28\") " pod="kserve/s3-init-qhnrw" Jul 31 23:53:45.754941 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.754925 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0277ebea-21b6-4021-b285-cee18ac86e28-model-cache\") pod \"s3-init-qhnrw\" (UID: \"0277ebea-21b6-4021-b285-cee18ac86e28\") " pod="kserve/s3-init-qhnrw" Jul 31 23:53:45.763572 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.763538 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xkkx\" (UniqueName: \"kubernetes.io/projected/0277ebea-21b6-4021-b285-cee18ac86e28-kube-api-access-9xkkx\") pod \"s3-init-qhnrw\" (UID: \"0277ebea-21b6-4021-b285-cee18ac86e28\") " pod="kserve/s3-init-qhnrw" Jul 31 23:53:45.853085 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.853011 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qhnrw" Jul 31 23:53:45.974729 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.974702 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-qhnrw"] Jul 31 23:53:45.977268 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:53:45.977238 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0277ebea_21b6_4021_b285_cee18ac86e28.slice/crio-cc737352b129078bd65f3b40275bf6bd0ec1856eda529983a9c2e988b490aaca WatchSource:0}: Error finding container cc737352b129078bd65f3b40275bf6bd0ec1856eda529983a9c2e988b490aaca: Status 404 returned error can't find the container with id cc737352b129078bd65f3b40275bf6bd0ec1856eda529983a9c2e988b490aaca Jul 31 23:53:45.979228 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:45.979207 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 23:53:46.179541 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:46.179454 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qhnrw" event={"ID":"0277ebea-21b6-4021-b285-cee18ac86e28","Type":"ContainerStarted","Data":"cc737352b129078bd65f3b40275bf6bd0ec1856eda529983a9c2e988b490aaca"} Jul 31 23:53:49.192857 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:49.192824 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qhnrw" event={"ID":"0277ebea-21b6-4021-b285-cee18ac86e28","Type":"ContainerStarted","Data":"9e2a313fe2a782d8be768444b90c0218dab0d3048b39bf0b419de5378f45b0bd"} Jul 31 23:53:51.202313 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:51.202278 2582 generic.go:356] "Generic (PLEG): container finished" podID="0277ebea-21b6-4021-b285-cee18ac86e28" containerID="9e2a313fe2a782d8be768444b90c0218dab0d3048b39bf0b419de5378f45b0bd" exitCode=0 Jul 31 23:53:51.202730 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:51.202357 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qhnrw" event={"ID":"0277ebea-21b6-4021-b285-cee18ac86e28","Type":"ContainerDied","Data":"9e2a313fe2a782d8be768444b90c0218dab0d3048b39bf0b419de5378f45b0bd"} Jul 31 23:53:56.227383 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:56.227343 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qhnrw" event={"ID":"0277ebea-21b6-4021-b285-cee18ac86e28","Type":"ContainerStarted","Data":"35d63239c4408d1fdad740e9b77f8ad02f2a9bdb5523ed40bb6fe23224179bb3"} Jul 31 23:53:56.244467 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:53:56.244384 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-qhnrw" podStartSLOduration=2.341669131 podStartE2EDuration="11.244368756s" totalImagesPullingTime="7.49228791s" totalInitContainerRuntime="1.410411715s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:53:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:53:56.243803688 +0000 UTC m=+453.403652801" watchObservedRunningTime="2026-07-31 23:53:56.244368756 +0000 UTC m=+453.404217868" Jul 31 23:55:01.468765 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:01.468728 2582 generic.go:356] "Generic (PLEG): container finished" podID="0277ebea-21b6-4021-b285-cee18ac86e28" containerID="35d63239c4408d1fdad740e9b77f8ad02f2a9bdb5523ed40bb6fe23224179bb3" exitCode=0 Jul 31 23:55:01.469189 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:01.468770 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qhnrw" event={"ID":"0277ebea-21b6-4021-b285-cee18ac86e28","Type":"ContainerDied","Data":"35d63239c4408d1fdad740e9b77f8ad02f2a9bdb5523ed40bb6fe23224179bb3"} Jul 31 23:55:02.594662 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:02.594639 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qhnrw" Jul 31 23:55:02.790570 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:02.790538 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xkkx\" (UniqueName: \"kubernetes.io/projected/0277ebea-21b6-4021-b285-cee18ac86e28-kube-api-access-9xkkx\") pod \"0277ebea-21b6-4021-b285-cee18ac86e28\" (UID: \"0277ebea-21b6-4021-b285-cee18ac86e28\") " Jul 31 23:55:02.790732 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:02.790601 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0277ebea-21b6-4021-b285-cee18ac86e28-model-cache\") pod \"0277ebea-21b6-4021-b285-cee18ac86e28\" (UID: \"0277ebea-21b6-4021-b285-cee18ac86e28\") " Jul 31 23:55:02.792793 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:02.792759 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0277ebea-21b6-4021-b285-cee18ac86e28-kube-api-access-9xkkx" (OuterVolumeSpecName: "kube-api-access-9xkkx") pod "0277ebea-21b6-4021-b285-cee18ac86e28" (UID: "0277ebea-21b6-4021-b285-cee18ac86e28"). InnerVolumeSpecName "kube-api-access-9xkkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:55:02.810146 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:02.810118 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0277ebea-21b6-4021-b285-cee18ac86e28-model-cache" (OuterVolumeSpecName: "model-cache") pod "0277ebea-21b6-4021-b285-cee18ac86e28" (UID: "0277ebea-21b6-4021-b285-cee18ac86e28"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:55:02.891923 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:02.891893 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xkkx\" (UniqueName: \"kubernetes.io/projected/0277ebea-21b6-4021-b285-cee18ac86e28-kube-api-access-9xkkx\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:55:02.891923 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:02.891922 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0277ebea-21b6-4021-b285-cee18ac86e28-model-cache\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:55:03.477471 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:03.477431 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qhnrw" event={"ID":"0277ebea-21b6-4021-b285-cee18ac86e28","Type":"ContainerDied","Data":"cc737352b129078bd65f3b40275bf6bd0ec1856eda529983a9c2e988b490aaca"} Jul 31 23:55:03.477471 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:03.477469 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qhnrw" Jul 31 23:55:03.477675 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:03.477472 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc737352b129078bd65f3b40275bf6bd0ec1856eda529983a9c2e988b490aaca" Jul 31 23:55:15.346685 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.346509 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk"] Jul 31 23:55:15.347241 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.347222 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0277ebea-21b6-4021-b285-cee18ac86e28" containerName="s3-init" Jul 31 23:55:15.347241 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.347242 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0277ebea-21b6-4021-b285-cee18ac86e28" containerName="s3-init" Jul 31 23:55:15.347344 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.347256 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0277ebea-21b6-4021-b285-cee18ac86e28" containerName="download-hf-model" Jul 31 23:55:15.347344 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.347262 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0277ebea-21b6-4021-b285-cee18ac86e28" containerName="download-hf-model" Jul 31 23:55:15.347344 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.347304 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0277ebea-21b6-4021-b285-cee18ac86e28" containerName="s3-init" Jul 31 23:55:15.351036 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.351013 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.352333 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.352310 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk"] Jul 31 23:55:15.355400 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.355379 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wctjk\"" Jul 31 23:55:15.355694 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.355502 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\"" Jul 31 23:55:15.356699 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.356673 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 31 23:55:15.357452 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.357431 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 31 23:55:15.357645 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.357504 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-predictor-serving-cert\"" Jul 31 23:55:15.357723 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.357551 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 31 23:55:15.357792 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.357558 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 31 23:55:15.392283 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.392257 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/420798ac-7982-48d0-a23a-0200a14f9d0a-proxy-tls\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.392396 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.392302 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-cabundle-cert\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.392396 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.392364 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/420798ac-7982-48d0-a23a-0200a14f9d0a-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.392536 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.392430 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plgjj\" (UniqueName: \"kubernetes.io/projected/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-api-access-plgjj\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.392536 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.392466 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.493059 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.492986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/420798ac-7982-48d0-a23a-0200a14f9d0a-proxy-tls\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.493059 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.493031 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-cabundle-cert\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.493059 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.493053 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/420798ac-7982-48d0-a23a-0200a14f9d0a-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.493300 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.493090 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plgjj\" (UniqueName: \"kubernetes.io/projected/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-api-access-plgjj\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.493300 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.493124 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.493300 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:55:15.493155 2582 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-canary-rollback-predictor-serving-cert: secret "isvc-canary-rollback-predictor-serving-cert" not found Jul 31 23:55:15.493300 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:55:15.493233 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/420798ac-7982-48d0-a23a-0200a14f9d0a-proxy-tls podName:420798ac-7982-48d0-a23a-0200a14f9d0a nodeName:}" failed. No retries permitted until 2026-07-31 23:55:15.993211919 +0000 UTC m=+533.153061009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/420798ac-7982-48d0-a23a-0200a14f9d0a-proxy-tls") pod "isvc-canary-rollback-predictor-8589bf8f95-5lrsk" (UID: "420798ac-7982-48d0-a23a-0200a14f9d0a") : secret "isvc-canary-rollback-predictor-serving-cert" not found Jul 31 23:55:15.493560 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.493538 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/420798ac-7982-48d0-a23a-0200a14f9d0a-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.493836 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.493816 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.493871 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.493816 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-cabundle-cert\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.501389 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.501363 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plgjj\" (UniqueName: \"kubernetes.io/projected/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-api-access-plgjj\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:15.997822 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:15.997791 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/420798ac-7982-48d0-a23a-0200a14f9d0a-proxy-tls\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:16.000155 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:16.000130 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/420798ac-7982-48d0-a23a-0200a14f9d0a-proxy-tls\") pod \"isvc-canary-rollback-predictor-8589bf8f95-5lrsk\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:16.265787 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:16.265699 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:16.391516 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:16.391489 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk"] Jul 31 23:55:16.394002 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:55:16.393976 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420798ac_7982_48d0_a23a_0200a14f9d0a.slice/crio-c6c9d314ead2a1c47e480a1859ce265dff9ea180da6bc38582f127fb6607bafc WatchSource:0}: Error finding container c6c9d314ead2a1c47e480a1859ce265dff9ea180da6bc38582f127fb6607bafc: Status 404 returned error can't find the container with id c6c9d314ead2a1c47e480a1859ce265dff9ea180da6bc38582f127fb6607bafc Jul 31 23:55:16.528037 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:16.527946 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" event={"ID":"420798ac-7982-48d0-a23a-0200a14f9d0a","Type":"ContainerStarted","Data":"6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c"} Jul 31 23:55:16.528037 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:16.528019 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" event={"ID":"420798ac-7982-48d0-a23a-0200a14f9d0a","Type":"ContainerStarted","Data":"c6c9d314ead2a1c47e480a1859ce265dff9ea180da6bc38582f127fb6607bafc"} Jul 31 23:55:20.542902 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:20.542865 2582 generic.go:356] "Generic (PLEG): container finished" podID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerID="6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c" exitCode=0 Jul 31 23:55:20.543268 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:20.542937 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" event={"ID":"420798ac-7982-48d0-a23a-0200a14f9d0a","Type":"ContainerDied","Data":"6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c"} Jul 31 23:55:33.611737 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:33.611650 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" event={"ID":"420798ac-7982-48d0-a23a-0200a14f9d0a","Type":"ContainerStarted","Data":"68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4"} Jul 31 23:55:35.622006 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:35.621978 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" event={"ID":"420798ac-7982-48d0-a23a-0200a14f9d0a","Type":"ContainerStarted","Data":"28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8"} Jul 31 23:55:35.622427 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:35.622113 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:35.645925 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:35.645880 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podStartSLOduration=2.084614489 podStartE2EDuration="20.645865831s" totalImagesPullingTime="14.913634695s" totalInitContainerRuntime="3.647616647s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:55:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:55:35.644588479 +0000 UTC m=+552.804437592" watchObservedRunningTime="2026-07-31 23:55:35.645865831 +0000 UTC m=+552.805714943" Jul 31 23:55:36.626724 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:36.626695 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:36.628095 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:36.628064 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 31 23:55:37.630004 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:37.629964 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 31 23:55:42.634531 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:42.634503 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:55:42.635052 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:42.635027 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 31 23:55:52.635889 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:55:52.635846 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 31 23:56:02.635732 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:56:02.635697 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 31 23:56:12.634927 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:56:12.634881 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 31 23:56:22.635729 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:56:22.635692 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 31 23:56:32.635455 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:56:32.635399 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 31 23:56:42.635914 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:56:42.635878 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 31 23:56:51.371027 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:56:51.370997 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:58:40.240921 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:40.240891 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk"] Jul 31 23:58:40.241395 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:40.241140 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" containerID="cri-o://68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4" gracePeriod=30 Jul 31 23:58:40.241395 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:40.241164 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kube-rbac-proxy" containerID="cri-o://28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8" gracePeriod=30 Jul 31 23:58:41.311295 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:41.311264 2582 generic.go:356] "Generic (PLEG): container finished" podID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerID="28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8" exitCode=2 Jul 31 23:58:41.311678 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:41.311336 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" event={"ID":"420798ac-7982-48d0-a23a-0200a14f9d0a","Type":"ContainerDied","Data":"28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8"} Jul 31 23:58:41.367534 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:41.367494 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 31 23:58:42.630392 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:42.630350 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.21:8643/healthz\": dial tcp 10.134.0.21:8643: connect: connection refused" Jul 31 23:58:44.682585 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.682560 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:58:44.763063 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.763029 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/420798ac-7982-48d0-a23a-0200a14f9d0a-proxy-tls\") pod \"420798ac-7982-48d0-a23a-0200a14f9d0a\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " Jul 31 23:58:44.763247 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.763097 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-rbac-proxy-sar-config\") pod \"420798ac-7982-48d0-a23a-0200a14f9d0a\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " Jul 31 23:58:44.763247 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.763134 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-cabundle-cert\") pod \"420798ac-7982-48d0-a23a-0200a14f9d0a\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " Jul 31 23:58:44.763247 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.763235 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plgjj\" (UniqueName: \"kubernetes.io/projected/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-api-access-plgjj\") pod \"420798ac-7982-48d0-a23a-0200a14f9d0a\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " Jul 31 23:58:44.763440 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.763283 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/420798ac-7982-48d0-a23a-0200a14f9d0a-kserve-provision-location\") pod \"420798ac-7982-48d0-a23a-0200a14f9d0a\" (UID: \"420798ac-7982-48d0-a23a-0200a14f9d0a\") " Jul 31 23:58:44.763504 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.763480 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "420798ac-7982-48d0-a23a-0200a14f9d0a" (UID: "420798ac-7982-48d0-a23a-0200a14f9d0a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:58:44.763559 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.763491 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "420798ac-7982-48d0-a23a-0200a14f9d0a" (UID: "420798ac-7982-48d0-a23a-0200a14f9d0a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 23:58:44.763675 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.763653 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/420798ac-7982-48d0-a23a-0200a14f9d0a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "420798ac-7982-48d0-a23a-0200a14f9d0a" (UID: "420798ac-7982-48d0-a23a-0200a14f9d0a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 23:58:44.763741 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.763723 2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-cabundle-cert\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:58:44.763778 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.763749 2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:58:44.765256 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.765235 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/420798ac-7982-48d0-a23a-0200a14f9d0a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "420798ac-7982-48d0-a23a-0200a14f9d0a" (UID: "420798ac-7982-48d0-a23a-0200a14f9d0a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 23:58:44.765341 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.765320 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-api-access-plgjj" (OuterVolumeSpecName: "kube-api-access-plgjj") pod "420798ac-7982-48d0-a23a-0200a14f9d0a" (UID: "420798ac-7982-48d0-a23a-0200a14f9d0a"). InnerVolumeSpecName "kube-api-access-plgjj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 23:58:44.864181 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.864102 2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/420798ac-7982-48d0-a23a-0200a14f9d0a-proxy-tls\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:58:44.864181 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.864131 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plgjj\" (UniqueName: \"kubernetes.io/projected/420798ac-7982-48d0-a23a-0200a14f9d0a-kube-api-access-plgjj\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:58:44.864181 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:44.864141 2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/420798ac-7982-48d0-a23a-0200a14f9d0a-kserve-provision-location\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Jul 31 23:58:45.328748 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.328709 2582 generic.go:356] "Generic (PLEG): container finished" podID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerID="68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4" exitCode=0 Jul 31 23:58:45.328979 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.328800 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" Jul 31 23:58:45.328979 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.328800 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" event={"ID":"420798ac-7982-48d0-a23a-0200a14f9d0a","Type":"ContainerDied","Data":"68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4"} Jul 31 23:58:45.328979 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.328906 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk" event={"ID":"420798ac-7982-48d0-a23a-0200a14f9d0a","Type":"ContainerDied","Data":"c6c9d314ead2a1c47e480a1859ce265dff9ea180da6bc38582f127fb6607bafc"} Jul 31 23:58:45.328979 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.328947 2582 scope.go:117] "RemoveContainer" containerID="28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8" Jul 31 23:58:45.338763 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.338722 2582 scope.go:117] "RemoveContainer" containerID="68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4" Jul 31 23:58:45.347849 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.347825 2582 scope.go:117] "RemoveContainer" containerID="6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c" Jul 31 23:58:45.355895 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.355871 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk"] Jul 31 23:58:45.356037 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.356023 2582 scope.go:117] "RemoveContainer" containerID="28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8" Jul 31 23:58:45.356317 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:58:45.356294 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8\": container with ID starting with 28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8 not found: ID does not exist" containerID="28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8" Jul 31 23:58:45.356359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.356330 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8"} err="failed to get container status \"28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8\": rpc error: code = NotFound desc = could not find container \"28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8\": container with ID starting with 28eaf7b68702d8fee3a60226eab30ce2ed80e264544a3626d0d3ffd68f6434e8 not found: ID does not exist" Jul 31 23:58:45.356359 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.356354 2582 scope.go:117] "RemoveContainer" containerID="68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4" Jul 31 23:58:45.356635 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:58:45.356616 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4\": container with ID starting with 68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4 not found: ID does not exist" containerID="68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4" Jul 31 23:58:45.356720 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.356644 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4"} err="failed to get container status \"68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4\": rpc error: code = NotFound desc = could not find container \"68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4\": container with ID starting with 68dfbc856520d0f65eef9ecfc11770398160ce45a6354da730c0eda342107be4 not found: ID does not exist" Jul 31 23:58:45.356720 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.356667 2582 scope.go:117] "RemoveContainer" containerID="6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c" Jul 31 23:58:45.356938 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:58:45.356918 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c\": container with ID starting with 6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c not found: ID does not exist" containerID="6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c" Jul 31 23:58:45.356988 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.356943 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c"} err="failed to get container status \"6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c\": rpc error: code = NotFound desc = could not find container \"6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c\": container with ID starting with 6f55fc33b1f91631794e394658010bd1fc861861b6b2bb6feb6df3ded3421d4c not found: ID does not exist" Jul 31 23:58:45.364304 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.364283 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-8589bf8f95-5lrsk"] Jul 31 23:58:45.372894 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.372870 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" path="/var/lib/kubelet/pods/420798ac-7982-48d0-a23a-0200a14f9d0a/volumes" Jul 31 23:58:45.436869 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.436843 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt"] Jul 31 23:58:45.437143 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.437129 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kube-rbac-proxy" Jul 31 23:58:45.437185 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.437146 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kube-rbac-proxy" Jul 31 23:58:45.437185 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.437159 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" Jul 31 23:58:45.437185 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.437164 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" Jul 31 23:58:45.437185 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.437178 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="storage-initializer" Jul 31 23:58:45.437185 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.437184 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="storage-initializer" Jul 31 23:58:45.437330 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.437240 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kserve-container" Jul 31 23:58:45.437330 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.437251 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="420798ac-7982-48d0-a23a-0200a14f9d0a" containerName="kube-rbac-proxy" Jul 31 23:58:45.441806 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.441784 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.444345 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.444322 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-predictor-serving-cert\"" Jul 31 23:58:45.444490 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.444396 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 31 23:58:45.444559 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.444524 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wctjk\"" Jul 31 23:58:45.444617 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.444580 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 31 23:58:45.444674 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.444636 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 31 23:58:45.444823 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.444807 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 31 23:58:45.444919 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.444894 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\"" Jul 31 23:58:45.450987 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.450965 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt"] Jul 31 23:58:45.568392 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.568351 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-cabundle-cert\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.568613 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.568452 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.568613 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.568494 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1be30840-115a-437d-b51e-0a6517e18701-kserve-provision-location\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.568613 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.568529 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm8s9\" (UniqueName: \"kubernetes.io/projected/1be30840-115a-437d-b51e-0a6517e18701-kube-api-access-dm8s9\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.568613 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.568592 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1be30840-115a-437d-b51e-0a6517e18701-proxy-tls\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.669655 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.669575 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-cabundle-cert\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.669655 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.669635 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.669853 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.669675 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1be30840-115a-437d-b51e-0a6517e18701-kserve-provision-location\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.669853 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.669707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm8s9\" (UniqueName: \"kubernetes.io/projected/1be30840-115a-437d-b51e-0a6517e18701-kube-api-access-dm8s9\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.669853 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.669739 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1be30840-115a-437d-b51e-0a6517e18701-proxy-tls\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.669996 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:58:45.669863 2582 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-canary-stop-predictor-serving-cert: secret "isvc-canary-stop-predictor-serving-cert" not found Jul 31 23:58:45.669996 ip-10-0-131-119 kubenswrapper[2582]: E0731 23:58:45.669935 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1be30840-115a-437d-b51e-0a6517e18701-proxy-tls podName:1be30840-115a-437d-b51e-0a6517e18701 nodeName:}" failed. No retries permitted until 2026-07-31 23:58:46.169912237 +0000 UTC m=+743.329761331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/1be30840-115a-437d-b51e-0a6517e18701-proxy-tls") pod "isvc-canary-stop-predictor-697f894f79-2zvrt" (UID: "1be30840-115a-437d-b51e-0a6517e18701") : secret "isvc-canary-stop-predictor-serving-cert" not found Jul 31 23:58:45.670158 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.670133 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1be30840-115a-437d-b51e-0a6517e18701-kserve-provision-location\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.670332 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.670314 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.670382 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.670365 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-cabundle-cert\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:45.680806 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:45.680787 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm8s9\" (UniqueName: \"kubernetes.io/projected/1be30840-115a-437d-b51e-0a6517e18701-kube-api-access-dm8s9\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:46.173781 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:46.173747 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1be30840-115a-437d-b51e-0a6517e18701-proxy-tls\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:46.176200 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:46.176176 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1be30840-115a-437d-b51e-0a6517e18701-proxy-tls\") pod \"isvc-canary-stop-predictor-697f894f79-2zvrt\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:46.351906 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:46.351874 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:46.481994 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:46.481968 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt"] Jul 31 23:58:46.484092 ip-10-0-131-119 kubenswrapper[2582]: W0731 23:58:46.484058 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be30840_115a_437d_b51e_0a6517e18701.slice/crio-c89a180fa16da262400d15dde6afb620abab5bf5a594264281dfc33ce6790282 WatchSource:0}: Error finding container c89a180fa16da262400d15dde6afb620abab5bf5a594264281dfc33ce6790282: Status 404 returned error can't find the container with id c89a180fa16da262400d15dde6afb620abab5bf5a594264281dfc33ce6790282 Jul 31 23:58:47.339482 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:47.339449 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" event={"ID":"1be30840-115a-437d-b51e-0a6517e18701","Type":"ContainerStarted","Data":"ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c"} Jul 31 23:58:47.339873 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:47.339502 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" event={"ID":"1be30840-115a-437d-b51e-0a6517e18701","Type":"ContainerStarted","Data":"c89a180fa16da262400d15dde6afb620abab5bf5a594264281dfc33ce6790282"} Jul 31 23:58:50.354447 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:50.354394 2582 generic.go:356] "Generic (PLEG): container finished" podID="1be30840-115a-437d-b51e-0a6517e18701" containerID="ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c" exitCode=0 Jul 31 23:58:50.354815 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:50.354451 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" event={"ID":"1be30840-115a-437d-b51e-0a6517e18701","Type":"ContainerDied","Data":"ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c"} Jul 31 23:58:51.360176 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:51.360140 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" event={"ID":"1be30840-115a-437d-b51e-0a6517e18701","Type":"ContainerStarted","Data":"7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea"} Jul 31 23:58:51.360592 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:51.360201 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" event={"ID":"1be30840-115a-437d-b51e-0a6517e18701","Type":"ContainerStarted","Data":"c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab"} Jul 31 23:58:51.360592 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:51.360389 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:51.377956 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:51.377902 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podStartSLOduration=2.7123223149999998 podStartE2EDuration="6.377883263s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.665560948s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:58:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:58:51.377429456 +0000 UTC m=+748.537278566" watchObservedRunningTime="2026-07-31 23:58:51.377883263 +0000 UTC m=+748.537732376" Jul 31 23:58:52.364467 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:52.364439 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:52.365780 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:52.365756 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Jul 31 23:58:53.368176 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:53.368137 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Jul 31 23:58:58.372579 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:58.372553 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Jul 31 23:58:58.373154 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:58:58.373012 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Jul 31 23:59:08.373904 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:59:08.373863 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Jul 31 23:59:18.375739 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:59:18.375675 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Jul 31 23:59:28.373601 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:59:28.373557 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Jul 31 23:59:38.373127 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:59:38.373083 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Jul 31 23:59:48.373891 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:59:48.373852 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Jul 31 23:59:58.373223 ip-10-0-131-119 kubenswrapper[2582]: I0731 23:59:58.373183 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Aug 01 00:00:01.366445 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:01.366389 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Aug 01 00:00:11.371079 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:11.371052 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Aug 01 00:00:15.405158 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:15.405129 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt"] Aug 01 00:00:15.405533 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:15.405381 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" containerID="cri-o://c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab" gracePeriod=30 Aug 01 00:00:15.405533 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:15.405461 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kube-rbac-proxy" containerID="cri-o://7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea" gracePeriod=30 Aug 01 00:00:15.667711 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:15.667625 2582 generic.go:356] "Generic (PLEG): container finished" podID="1be30840-115a-437d-b51e-0a6517e18701" containerID="7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea" exitCode=2 Aug 01 00:00:15.667711 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:15.667669 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" event={"ID":"1be30840-115a-437d-b51e-0a6517e18701","Type":"ContainerDied","Data":"7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea"} Aug 01 00:00:18.369187 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:18.369137 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.22:8643/healthz\": dial tcp 10.134.0.22:8643: connect: connection refused" Aug 01 00:00:20.242577 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.242555 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Aug 01 00:00:20.348312 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.348235 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-kube-rbac-proxy-sar-config\") pod \"1be30840-115a-437d-b51e-0a6517e18701\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " Aug 01 00:00:20.348312 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.348284 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1be30840-115a-437d-b51e-0a6517e18701-kserve-provision-location\") pod \"1be30840-115a-437d-b51e-0a6517e18701\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " Aug 01 00:00:20.348312 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.348310 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1be30840-115a-437d-b51e-0a6517e18701-proxy-tls\") pod \"1be30840-115a-437d-b51e-0a6517e18701\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " Aug 01 00:00:20.348579 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.348363 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-cabundle-cert\") pod \"1be30840-115a-437d-b51e-0a6517e18701\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " Aug 01 00:00:20.348579 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.348384 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm8s9\" (UniqueName: \"kubernetes.io/projected/1be30840-115a-437d-b51e-0a6517e18701-kube-api-access-dm8s9\") pod \"1be30840-115a-437d-b51e-0a6517e18701\" (UID: \"1be30840-115a-437d-b51e-0a6517e18701\") " Aug 01 00:00:20.348678 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.348610 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1be30840-115a-437d-b51e-0a6517e18701-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1be30840-115a-437d-b51e-0a6517e18701" (UID: "1be30840-115a-437d-b51e-0a6517e18701"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 01 00:00:20.348732 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.348666 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1be30840-115a-437d-b51e-0a6517e18701" (UID: "1be30840-115a-437d-b51e-0a6517e18701"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 01 00:00:20.348818 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.348728 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "1be30840-115a-437d-b51e-0a6517e18701" (UID: "1be30840-115a-437d-b51e-0a6517e18701"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 01 00:00:20.350460 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.350436 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1be30840-115a-437d-b51e-0a6517e18701-kube-api-access-dm8s9" (OuterVolumeSpecName: "kube-api-access-dm8s9") pod "1be30840-115a-437d-b51e-0a6517e18701" (UID: "1be30840-115a-437d-b51e-0a6517e18701"). InnerVolumeSpecName "kube-api-access-dm8s9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 01 00:00:20.350544 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.350523 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1be30840-115a-437d-b51e-0a6517e18701-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1be30840-115a-437d-b51e-0a6517e18701" (UID: "1be30840-115a-437d-b51e-0a6517e18701"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 01 00:00:20.449902 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.449867 2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1be30840-115a-437d-b51e-0a6517e18701-proxy-tls\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:00:20.449902 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.449895 2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-cabundle-cert\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:00:20.449902 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.449905 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm8s9\" (UniqueName: \"kubernetes.io/projected/1be30840-115a-437d-b51e-0a6517e18701-kube-api-access-dm8s9\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:00:20.450131 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.449914 2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1be30840-115a-437d-b51e-0a6517e18701-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:00:20.450131 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.449923 2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1be30840-115a-437d-b51e-0a6517e18701-kserve-provision-location\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:00:20.689048 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.688950 2582 generic.go:356] "Generic (PLEG): container finished" podID="1be30840-115a-437d-b51e-0a6517e18701" containerID="c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab" exitCode=0 Aug 01 00:00:20.689048 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.689005 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" event={"ID":"1be30840-115a-437d-b51e-0a6517e18701","Type":"ContainerDied","Data":"c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab"} Aug 01 00:00:20.689048 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.689021 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" Aug 01 00:00:20.689296 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.689069 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt" event={"ID":"1be30840-115a-437d-b51e-0a6517e18701","Type":"ContainerDied","Data":"c89a180fa16da262400d15dde6afb620abab5bf5a594264281dfc33ce6790282"} Aug 01 00:00:20.689296 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.689106 2582 scope.go:117] "RemoveContainer" containerID="7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea" Aug 01 00:00:20.698478 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.698457 2582 scope.go:117] "RemoveContainer" containerID="c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab" Aug 01 00:00:20.706799 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.706773 2582 scope.go:117] "RemoveContainer" containerID="ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c" Aug 01 00:00:20.712733 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.712704 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt"] Aug 01 00:00:20.714691 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.714672 2582 scope.go:117] "RemoveContainer" containerID="7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea" Aug 01 00:00:20.714966 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:00:20.714947 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea\": container with ID starting with 7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea not found: ID does not exist" containerID="7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea" Aug 01 00:00:20.715028 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.714973 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea"} err="failed to get container status \"7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea\": rpc error: code = NotFound desc = could not find container \"7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea\": container with ID starting with 7dae2561462bf3f28ae3961dd2420d7ff1dad29f89be974779c1e4826dd868ea not found: ID does not exist" Aug 01 00:00:20.715028 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.714992 2582 scope.go:117] "RemoveContainer" containerID="c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab" Aug 01 00:00:20.715236 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:00:20.715220 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab\": container with ID starting with c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab not found: ID does not exist" containerID="c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab" Aug 01 00:00:20.715294 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.715240 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab"} err="failed to get container status \"c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab\": rpc error: code = NotFound desc = could not find container \"c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab\": container with ID starting with c45b041e0baf63526eac2e338a1df7fba2562dfee4ab83a5aa6afd3b40319aab not found: ID does not exist" Aug 01 00:00:20.715294 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.715253 2582 scope.go:117] "RemoveContainer" containerID="ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c" Aug 01 00:00:20.715514 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:00:20.715492 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c\": container with ID starting with ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c not found: ID does not exist" containerID="ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c" Aug 01 00:00:20.715564 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.715523 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c"} err="failed to get container status \"ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c\": rpc error: code = NotFound desc = could not find container \"ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c\": container with ID starting with ea71e1b0a9d57123a85722d9aa2dc4ebab7c8ad04876ca06c47039affec1353c not found: ID does not exist" Aug 01 00:00:20.719589 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:20.719568 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-697f894f79-2zvrt"] Aug 01 00:00:21.371147 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:21.371116 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1be30840-115a-437d-b51e-0a6517e18701" path="/var/lib/kubelet/pods/1be30840-115a-437d-b51e-0a6517e18701/volumes" Aug 01 00:00:25.699241 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.699205 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2"] Aug 01 00:00:25.699616 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.699512 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="storage-initializer" Aug 01 00:00:25.699616 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.699525 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="storage-initializer" Aug 01 00:00:25.699616 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.699534 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" Aug 01 00:00:25.699616 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.699540 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" Aug 01 00:00:25.699616 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.699554 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kube-rbac-proxy" Aug 01 00:00:25.699616 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.699559 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kube-rbac-proxy" Aug 01 00:00:25.699616 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.699600 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kube-rbac-proxy" Aug 01 00:00:25.699616 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.699608 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1be30840-115a-437d-b51e-0a6517e18701" containerName="kserve-container" Aug 01 00:00:25.704000 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.703972 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.706798 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.706772 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-53f567-kube-rbac-proxy-sar-config\"" Aug 01 00:00:25.706908 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.706799 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wctjk\"" Aug 01 00:00:25.706908 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.706811 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Aug 01 00:00:25.706908 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.706856 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Aug 01 00:00:25.706908 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.706813 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Aug 01 00:00:25.707668 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.707651 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-53f567-predictor-serving-cert\"" Aug 01 00:00:25.707734 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.707703 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Aug 01 00:00:25.713316 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.713293 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2"] Aug 01 00:00:25.792559 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.792531 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7tpr\" (UniqueName: \"kubernetes.io/projected/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-api-access-t7tpr\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.792697 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.792598 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-cabundle-cert\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.792697 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.792665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83f86e36-36ec-4d6f-91dc-cb97472854e2-proxy-tls\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.792808 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.792697 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-rbac-proxy-sar-config\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.792808 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.792781 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83f86e36-36ec-4d6f-91dc-cb97472854e2-kserve-provision-location\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.893399 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.893365 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83f86e36-36ec-4d6f-91dc-cb97472854e2-proxy-tls\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.893564 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.893429 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-rbac-proxy-sar-config\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.893564 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.893484 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83f86e36-36ec-4d6f-91dc-cb97472854e2-kserve-provision-location\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.893564 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:00:25.893513 2582 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-primary-53f567-predictor-serving-cert: secret "isvc-primary-53f567-predictor-serving-cert" not found Aug 01 00:00:25.893564 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.893530 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7tpr\" (UniqueName: \"kubernetes.io/projected/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-api-access-t7tpr\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.893564 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.893560 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-cabundle-cert\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.893790 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:00:25.893586 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83f86e36-36ec-4d6f-91dc-cb97472854e2-proxy-tls podName:83f86e36-36ec-4d6f-91dc-cb97472854e2 nodeName:}" failed. No retries permitted until 2026-08-01 00:00:26.393564265 +0000 UTC m=+843.553413355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/83f86e36-36ec-4d6f-91dc-cb97472854e2-proxy-tls") pod "isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" (UID: "83f86e36-36ec-4d6f-91dc-cb97472854e2") : secret "isvc-primary-53f567-predictor-serving-cert" not found Aug 01 00:00:25.894103 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.894080 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83f86e36-36ec-4d6f-91dc-cb97472854e2-kserve-provision-location\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.894233 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.894213 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-cabundle-cert\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.894275 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.894213 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-rbac-proxy-sar-config\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:25.901770 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:25.901749 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7tpr\" (UniqueName: \"kubernetes.io/projected/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-api-access-t7tpr\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:26.397957 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:26.397923 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83f86e36-36ec-4d6f-91dc-cb97472854e2-proxy-tls\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:26.400231 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:26.400204 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83f86e36-36ec-4d6f-91dc-cb97472854e2-proxy-tls\") pod \"isvc-primary-53f567-predictor-748cdd4dc9-vfrr2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:26.615514 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:26.615479 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:26.750463 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:26.750433 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2"] Aug 01 00:00:26.754456 ip-10-0-131-119 kubenswrapper[2582]: W0801 00:00:26.754432 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83f86e36_36ec_4d6f_91dc_cb97472854e2.slice/crio-6ad80a836a2265f3c04d7cf8cc93b63b446bd28c782535cde0a8fa6a4e198146 WatchSource:0}: Error finding container 6ad80a836a2265f3c04d7cf8cc93b63b446bd28c782535cde0a8fa6a4e198146: Status 404 returned error can't find the container with id 6ad80a836a2265f3c04d7cf8cc93b63b446bd28c782535cde0a8fa6a4e198146 Aug 01 00:00:27.717520 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:27.717479 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" event={"ID":"83f86e36-36ec-4d6f-91dc-cb97472854e2","Type":"ContainerStarted","Data":"09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3"} Aug 01 00:00:27.717718 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:27.717547 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" event={"ID":"83f86e36-36ec-4d6f-91dc-cb97472854e2","Type":"ContainerStarted","Data":"6ad80a836a2265f3c04d7cf8cc93b63b446bd28c782535cde0a8fa6a4e198146"} Aug 01 00:00:30.731981 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:30.731950 2582 generic.go:356] "Generic (PLEG): container finished" podID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerID="09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3" exitCode=0 Aug 01 00:00:30.732352 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:30.732025 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" event={"ID":"83f86e36-36ec-4d6f-91dc-cb97472854e2","Type":"ContainerDied","Data":"09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3"} Aug 01 00:00:31.739619 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:31.739584 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" event={"ID":"83f86e36-36ec-4d6f-91dc-cb97472854e2","Type":"ContainerStarted","Data":"94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8"} Aug 01 00:00:31.740061 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:31.739659 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" event={"ID":"83f86e36-36ec-4d6f-91dc-cb97472854e2","Type":"ContainerStarted","Data":"08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420"} Aug 01 00:00:31.740061 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:31.739938 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:31.758858 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:31.758805 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podStartSLOduration=2.949246196 podStartE2EDuration="6.758788026s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.80954183s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:00:31.757581432 +0000 UTC m=+848.917430569" watchObservedRunningTime="2026-08-01 00:00:31.758788026 +0000 UTC m=+848.918637137" Aug 01 00:00:32.743910 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:32.743876 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:32.745312 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:32.745285 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Aug 01 00:00:33.747292 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:33.747248 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Aug 01 00:00:38.752235 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:38.752209 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:00:38.752754 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:38.752716 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Aug 01 00:00:48.752718 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:48.752679 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Aug 01 00:00:58.752907 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:00:58.752870 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Aug 01 00:01:08.752990 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:01:08.752953 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Aug 01 00:01:18.752788 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:01:18.752693 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Aug 01 00:01:28.753504 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:01:28.753461 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Aug 01 00:01:38.753268 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:01:38.753229 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Aug 01 00:01:48.753727 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:01:48.753694 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:02:28.944136 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:28.944098 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2"] Aug 01 00:02:28.946922 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:28.944389 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" containerID="cri-o://08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420" gracePeriod=30 Aug 01 00:02:28.946922 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:28.944482 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kube-rbac-proxy" containerID="cri-o://94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8" gracePeriod=30 Aug 01 00:02:29.178629 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:29.178598 2582 generic.go:356] "Generic (PLEG): container finished" podID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerID="94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8" exitCode=2 Aug 01 00:02:29.178790 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:29.178667 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" event={"ID":"83f86e36-36ec-4d6f-91dc-cb97472854e2","Type":"ContainerDied","Data":"94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8"} Aug 01 00:02:33.584386 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.584359 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:02:33.659004 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.658973 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-cabundle-cert\") pod \"83f86e36-36ec-4d6f-91dc-cb97472854e2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " Aug 01 00:02:33.659004 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.659008 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83f86e36-36ec-4d6f-91dc-cb97472854e2-proxy-tls\") pod \"83f86e36-36ec-4d6f-91dc-cb97472854e2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " Aug 01 00:02:33.659204 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.659027 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-rbac-proxy-sar-config\") pod \"83f86e36-36ec-4d6f-91dc-cb97472854e2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " Aug 01 00:02:33.659204 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.659076 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83f86e36-36ec-4d6f-91dc-cb97472854e2-kserve-provision-location\") pod \"83f86e36-36ec-4d6f-91dc-cb97472854e2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " Aug 01 00:02:33.659277 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.659247 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7tpr\" (UniqueName: \"kubernetes.io/projected/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-api-access-t7tpr\") pod \"83f86e36-36ec-4d6f-91dc-cb97472854e2\" (UID: \"83f86e36-36ec-4d6f-91dc-cb97472854e2\") " Aug 01 00:02:33.659427 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.659382 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f86e36-36ec-4d6f-91dc-cb97472854e2-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "83f86e36-36ec-4d6f-91dc-cb97472854e2" (UID: "83f86e36-36ec-4d6f-91dc-cb97472854e2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 01 00:02:33.659508 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.659448 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "83f86e36-36ec-4d6f-91dc-cb97472854e2" (UID: "83f86e36-36ec-4d6f-91dc-cb97472854e2"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 01 00:02:33.659508 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.659458 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "83f86e36-36ec-4d6f-91dc-cb97472854e2" (UID: "83f86e36-36ec-4d6f-91dc-cb97472854e2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 01 00:02:33.659602 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.659556 2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-cabundle-cert\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:02:33.659602 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.659571 2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:02:33.659602 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.659581 2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83f86e36-36ec-4d6f-91dc-cb97472854e2-kserve-provision-location\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:02:33.661167 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.661147 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83f86e36-36ec-4d6f-91dc-cb97472854e2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "83f86e36-36ec-4d6f-91dc-cb97472854e2" (UID: "83f86e36-36ec-4d6f-91dc-cb97472854e2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 01 00:02:33.661281 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.661260 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-api-access-t7tpr" (OuterVolumeSpecName: "kube-api-access-t7tpr") pod "83f86e36-36ec-4d6f-91dc-cb97472854e2" (UID: "83f86e36-36ec-4d6f-91dc-cb97472854e2"). InnerVolumeSpecName "kube-api-access-t7tpr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 01 00:02:33.759907 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.759877 2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83f86e36-36ec-4d6f-91dc-cb97472854e2-proxy-tls\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:02:33.759907 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:33.759905 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7tpr\" (UniqueName: \"kubernetes.io/projected/83f86e36-36ec-4d6f-91dc-cb97472854e2-kube-api-access-t7tpr\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:02:34.149874 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.149789 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk"] Aug 01 00:02:34.150246 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.150233 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" Aug 01 00:02:34.150289 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.150250 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" Aug 01 00:02:34.150289 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.150278 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kube-rbac-proxy" Aug 01 00:02:34.150289 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.150286 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kube-rbac-proxy" Aug 01 00:02:34.150392 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.150309 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="storage-initializer" Aug 01 00:02:34.150392 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.150318 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="storage-initializer" Aug 01 00:02:34.150392 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.150375 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kube-rbac-proxy" Aug 01 00:02:34.150392 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.150386 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerName="kserve-container" Aug 01 00:02:34.154478 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.154458 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.158605 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.158583 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-init-fail-168342-predictor-serving-cert\"" Aug 01 00:02:34.158990 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.158970 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"fail-s3-sa-168342-dockercfg-jd62m\"" Aug 01 00:02:34.159153 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.159028 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-init-fail-168342-kube-rbac-proxy-sar-config\"" Aug 01 00:02:34.159153 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.159032 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"fail-s3-secret-168342\"" Aug 01 00:02:34.167964 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.167940 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk"] Aug 01 00:02:34.199573 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.199548 2582 generic.go:356] "Generic (PLEG): container finished" podID="83f86e36-36ec-4d6f-91dc-cb97472854e2" containerID="08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420" exitCode=0 Aug 01 00:02:34.199700 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.199640 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" Aug 01 00:02:34.199700 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.199638 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" event={"ID":"83f86e36-36ec-4d6f-91dc-cb97472854e2","Type":"ContainerDied","Data":"08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420"} Aug 01 00:02:34.199820 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.199708 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2" event={"ID":"83f86e36-36ec-4d6f-91dc-cb97472854e2","Type":"ContainerDied","Data":"6ad80a836a2265f3c04d7cf8cc93b63b446bd28c782535cde0a8fa6a4e198146"} Aug 01 00:02:34.199820 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.199749 2582 scope.go:117] "RemoveContainer" containerID="94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8" Aug 01 00:02:34.207603 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.207587 2582 scope.go:117] "RemoveContainer" containerID="08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420" Aug 01 00:02:34.214611 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.214594 2582 scope.go:117] "RemoveContainer" containerID="09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3" Aug 01 00:02:34.221182 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.221165 2582 scope.go:117] "RemoveContainer" containerID="94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8" Aug 01 00:02:34.221473 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:02:34.221419 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8\": container with ID starting with 94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8 not found: ID does not exist" containerID="94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8" Aug 01 00:02:34.221574 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.221479 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8"} err="failed to get container status \"94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8\": rpc error: code = NotFound desc = could not find container \"94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8\": container with ID starting with 94808ccbda441c03c14e6fd8b4a075810e60ba6e437f716c8b9d9d1dc9f05fe8 not found: ID does not exist" Aug 01 00:02:34.221574 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.221503 2582 scope.go:117] "RemoveContainer" containerID="08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420" Aug 01 00:02:34.221748 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:02:34.221732 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420\": container with ID starting with 08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420 not found: ID does not exist" containerID="08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420" Aug 01 00:02:34.221788 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.221752 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420"} err="failed to get container status \"08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420\": rpc error: code = NotFound desc = could not find container \"08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420\": container with ID starting with 08462f305e502c1ed7b9c51a5959136671bb64156796984595066fe6e2d9f420 not found: ID does not exist" Aug 01 00:02:34.221788 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.221766 2582 scope.go:117] "RemoveContainer" containerID="09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3" Aug 01 00:02:34.221958 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:02:34.221943 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3\": container with ID starting with 09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3 not found: ID does not exist" containerID="09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3" Aug 01 00:02:34.222001 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.221962 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3"} err="failed to get container status \"09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3\": rpc error: code = NotFound desc = could not find container \"09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3\": container with ID starting with 09cd282bb6134f691f9b774d4eb4526f400091c4ce5319a36b37c5b19d26f0d3 not found: ID does not exist" Aug 01 00:02:34.224343 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.224323 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2"] Aug 01 00:02:34.227994 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.227973 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-53f567-predictor-748cdd4dc9-vfrr2"] Aug 01 00:02:34.264178 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.264153 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20d63252-f064-45b3-9f92-b01142043e70-kserve-provision-location\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.264268 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.264184 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pff2\" (UniqueName: \"kubernetes.io/projected/20d63252-f064-45b3-9f92-b01142043e70-kube-api-access-9pff2\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.264268 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.264208 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-cabundle-cert\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.264342 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.264298 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.264342 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.264320 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20d63252-f064-45b3-9f92-b01142043e70-proxy-tls\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.365334 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.365308 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20d63252-f064-45b3-9f92-b01142043e70-kserve-provision-location\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.365471 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.365340 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pff2\" (UniqueName: \"kubernetes.io/projected/20d63252-f064-45b3-9f92-b01142043e70-kube-api-access-9pff2\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.365471 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.365362 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-cabundle-cert\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.365471 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.365428 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.365471 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.365447 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20d63252-f064-45b3-9f92-b01142043e70-proxy-tls\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.365683 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:02:34.365548 2582 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-init-fail-168342-predictor-serving-cert: secret "isvc-init-fail-168342-predictor-serving-cert" not found Aug 01 00:02:34.365683 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:02:34.365604 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20d63252-f064-45b3-9f92-b01142043e70-proxy-tls podName:20d63252-f064-45b3-9f92-b01142043e70 nodeName:}" failed. No retries permitted until 2026-08-01 00:02:34.865585198 +0000 UTC m=+972.025434299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/20d63252-f064-45b3-9f92-b01142043e70-proxy-tls") pod "isvc-init-fail-168342-predictor-5dd6479db6-kthxk" (UID: "20d63252-f064-45b3-9f92-b01142043e70") : secret "isvc-init-fail-168342-predictor-serving-cert" not found Aug 01 00:02:34.365793 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.365754 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20d63252-f064-45b3-9f92-b01142043e70-kserve-provision-location\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.366046 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.366028 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-cabundle-cert\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.366157 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.366135 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.374910 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.374891 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pff2\" (UniqueName: \"kubernetes.io/projected/20d63252-f064-45b3-9f92-b01142043e70-kube-api-access-9pff2\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.869628 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.869593 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20d63252-f064-45b3-9f92-b01142043e70-proxy-tls\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:34.871972 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:34.871953 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20d63252-f064-45b3-9f92-b01142043e70-proxy-tls\") pod \"isvc-init-fail-168342-predictor-5dd6479db6-kthxk\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:35.065832 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:35.065795 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:35.193141 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:35.193109 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk"] Aug 01 00:02:35.195942 ip-10-0-131-119 kubenswrapper[2582]: W0801 00:02:35.195916 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d63252_f064_45b3_9f92_b01142043e70.slice/crio-2f6a79e91fef7f9ea1f51d0378864e6fc14b711f6b8dd66522b96a30b2da80ec WatchSource:0}: Error finding container 2f6a79e91fef7f9ea1f51d0378864e6fc14b711f6b8dd66522b96a30b2da80ec: Status 404 returned error can't find the container with id 2f6a79e91fef7f9ea1f51d0378864e6fc14b711f6b8dd66522b96a30b2da80ec Aug 01 00:02:35.205128 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:35.205102 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" event={"ID":"20d63252-f064-45b3-9f92-b01142043e70","Type":"ContainerStarted","Data":"2f6a79e91fef7f9ea1f51d0378864e6fc14b711f6b8dd66522b96a30b2da80ec"} Aug 01 00:02:35.371236 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:35.371205 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83f86e36-36ec-4d6f-91dc-cb97472854e2" path="/var/lib/kubelet/pods/83f86e36-36ec-4d6f-91dc-cb97472854e2/volumes" Aug 01 00:02:36.211642 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:36.211599 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" event={"ID":"20d63252-f064-45b3-9f92-b01142043e70","Type":"ContainerStarted","Data":"2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f"} Aug 01 00:02:38.221128 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:38.221101 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-168342-predictor-5dd6479db6-kthxk_20d63252-f064-45b3-9f92-b01142043e70/storage-initializer/0.log" Aug 01 00:02:38.221516 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:38.221138 2582 generic.go:356] "Generic (PLEG): container finished" podID="20d63252-f064-45b3-9f92-b01142043e70" containerID="2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f" exitCode=1 Aug 01 00:02:38.221516 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:38.221216 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" event={"ID":"20d63252-f064-45b3-9f92-b01142043e70","Type":"ContainerDied","Data":"2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f"} Aug 01 00:02:39.120985 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:39.120951 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk"] Aug 01 00:02:39.226468 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:39.226441 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-168342-predictor-5dd6479db6-kthxk_20d63252-f064-45b3-9f92-b01142043e70/storage-initializer/0.log" Aug 01 00:02:39.226885 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:39.226498 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" event={"ID":"20d63252-f064-45b3-9f92-b01142043e70","Type":"ContainerStarted","Data":"75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02"} Aug 01 00:02:39.226885 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:39.226662 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" podUID="20d63252-f064-45b3-9f92-b01142043e70" containerName="storage-initializer" containerID="cri-o://75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02" gracePeriod=30 Aug 01 00:02:42.064098 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.064076 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-168342-predictor-5dd6479db6-kthxk_20d63252-f064-45b3-9f92-b01142043e70/storage-initializer/1.log" Aug 01 00:02:42.064471 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.064457 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-168342-predictor-5dd6479db6-kthxk_20d63252-f064-45b3-9f92-b01142043e70/storage-initializer/0.log" Aug 01 00:02:42.066652 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.066638 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:42.232027 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.232001 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-kube-rbac-proxy-sar-config\") pod \"20d63252-f064-45b3-9f92-b01142043e70\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " Aug 01 00:02:42.232164 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.232041 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-cabundle-cert\") pod \"20d63252-f064-45b3-9f92-b01142043e70\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " Aug 01 00:02:42.232164 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.232086 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pff2\" (UniqueName: \"kubernetes.io/projected/20d63252-f064-45b3-9f92-b01142043e70-kube-api-access-9pff2\") pod \"20d63252-f064-45b3-9f92-b01142043e70\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " Aug 01 00:02:42.232164 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.232129 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20d63252-f064-45b3-9f92-b01142043e70-proxy-tls\") pod \"20d63252-f064-45b3-9f92-b01142043e70\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " Aug 01 00:02:42.232295 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.232204 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20d63252-f064-45b3-9f92-b01142043e70-kserve-provision-location\") pod \"20d63252-f064-45b3-9f92-b01142043e70\" (UID: \"20d63252-f064-45b3-9f92-b01142043e70\") " Aug 01 00:02:42.232533 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.232502 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20d63252-f064-45b3-9f92-b01142043e70-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "20d63252-f064-45b3-9f92-b01142043e70" (UID: "20d63252-f064-45b3-9f92-b01142043e70"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 01 00:02:42.232629 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.232539 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "20d63252-f064-45b3-9f92-b01142043e70" (UID: "20d63252-f064-45b3-9f92-b01142043e70"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 01 00:02:42.232629 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.232566 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "20d63252-f064-45b3-9f92-b01142043e70" (UID: "20d63252-f064-45b3-9f92-b01142043e70"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 01 00:02:42.234147 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.234126 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20d63252-f064-45b3-9f92-b01142043e70-kube-api-access-9pff2" (OuterVolumeSpecName: "kube-api-access-9pff2") pod "20d63252-f064-45b3-9f92-b01142043e70" (UID: "20d63252-f064-45b3-9f92-b01142043e70"). InnerVolumeSpecName "kube-api-access-9pff2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 01 00:02:42.234324 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.234299 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20d63252-f064-45b3-9f92-b01142043e70-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "20d63252-f064-45b3-9f92-b01142043e70" (UID: "20d63252-f064-45b3-9f92-b01142043e70"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 01 00:02:42.240269 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.240249 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-168342-predictor-5dd6479db6-kthxk_20d63252-f064-45b3-9f92-b01142043e70/storage-initializer/1.log" Aug 01 00:02:42.240724 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.240711 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-168342-predictor-5dd6479db6-kthxk_20d63252-f064-45b3-9f92-b01142043e70/storage-initializer/0.log" Aug 01 00:02:42.240805 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.240752 2582 generic.go:356] "Generic (PLEG): container finished" podID="20d63252-f064-45b3-9f92-b01142043e70" containerID="75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02" exitCode=1 Aug 01 00:02:42.240857 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.240810 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" Aug 01 00:02:42.240909 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.240844 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" event={"ID":"20d63252-f064-45b3-9f92-b01142043e70","Type":"ContainerDied","Data":"75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02"} Aug 01 00:02:42.240963 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.240920 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk" event={"ID":"20d63252-f064-45b3-9f92-b01142043e70","Type":"ContainerDied","Data":"2f6a79e91fef7f9ea1f51d0378864e6fc14b711f6b8dd66522b96a30b2da80ec"} Aug 01 00:02:42.241014 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.240964 2582 scope.go:117] "RemoveContainer" containerID="75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02" Aug 01 00:02:42.249476 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.249457 2582 scope.go:117] "RemoveContainer" containerID="2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f" Aug 01 00:02:42.256500 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.256482 2582 scope.go:117] "RemoveContainer" containerID="75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02" Aug 01 00:02:42.256746 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:02:42.256721 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02\": container with ID starting with 75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02 not found: ID does not exist" containerID="75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02" Aug 01 00:02:42.256809 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.256758 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02"} err="failed to get container status \"75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02\": rpc error: code = NotFound desc = could not find container \"75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02\": container with ID starting with 75a123b8cf87572e3b019510e86b7f8ecd8a70e8eacd1226660c8f9bc24a5b02 not found: ID does not exist" Aug 01 00:02:42.256809 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.256782 2582 scope.go:117] "RemoveContainer" containerID="2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f" Aug 01 00:02:42.257044 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:02:42.257026 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f\": container with ID starting with 2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f not found: ID does not exist" containerID="2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f" Aug 01 00:02:42.257088 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.257050 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f"} err="failed to get container status \"2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f\": rpc error: code = NotFound desc = could not find container \"2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f\": container with ID starting with 2e53b50aa7bfa1163c588866a57b18279a2eb9b5bdd9dac9b23a31bcbd54751f not found: ID does not exist" Aug 01 00:02:42.274803 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.274782 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk"] Aug 01 00:02:42.280924 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.280896 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-168342-predictor-5dd6479db6-kthxk"] Aug 01 00:02:42.333860 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.333824 2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20d63252-f064-45b3-9f92-b01142043e70-kserve-provision-location\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:02:42.333860 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.333854 2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:02:42.334049 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.333870 2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/20d63252-f064-45b3-9f92-b01142043e70-cabundle-cert\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:02:42.334049 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.333883 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pff2\" (UniqueName: \"kubernetes.io/projected/20d63252-f064-45b3-9f92-b01142043e70-kube-api-access-9pff2\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:02:42.334049 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:42.333895 2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20d63252-f064-45b3-9f92-b01142043e70-proxy-tls\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:02:43.373436 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:02:43.372899 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20d63252-f064-45b3-9f92-b01142043e70" path="/var/lib/kubelet/pods/20d63252-f064-45b3-9f92-b01142043e70/volumes" Aug 01 00:12:07.198846 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.198806 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-brn9x/must-gather-svz8j"] Aug 01 00:12:07.199317 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.199124 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20d63252-f064-45b3-9f92-b01142043e70" containerName="storage-initializer" Aug 01 00:12:07.199317 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.199135 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="20d63252-f064-45b3-9f92-b01142043e70" containerName="storage-initializer" Aug 01 00:12:07.199317 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.199182 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="20d63252-f064-45b3-9f92-b01142043e70" containerName="storage-initializer" Aug 01 00:12:07.199317 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.199190 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="20d63252-f064-45b3-9f92-b01142043e70" containerName="storage-initializer" Aug 01 00:12:07.199317 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.199234 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20d63252-f064-45b3-9f92-b01142043e70" containerName="storage-initializer" Aug 01 00:12:07.199317 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.199239 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="20d63252-f064-45b3-9f92-b01142043e70" containerName="storage-initializer" Aug 01 00:12:07.201671 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.201651 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:07.204585 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.204559 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-brn9x\"/\"default-dockercfg-2qfk4\"" Aug 01 00:12:07.204792 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.204765 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-brn9x\"/\"kube-root-ca.crt\"" Aug 01 00:12:07.204889 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.204858 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-brn9x\"/\"openshift-service-ca.crt\"" Aug 01 00:12:07.210159 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.210138 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-brn9x/must-gather-svz8j"] Aug 01 00:12:07.212853 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.212834 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnbdq\" (UniqueName: \"kubernetes.io/projected/dfe559be-dba3-40b1-a187-6bb419f2410a-kube-api-access-mnbdq\") pod \"must-gather-svz8j\" (UID: \"dfe559be-dba3-40b1-a187-6bb419f2410a\") " pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:07.212962 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.212900 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dfe559be-dba3-40b1-a187-6bb419f2410a-must-gather-output\") pod \"must-gather-svz8j\" (UID: \"dfe559be-dba3-40b1-a187-6bb419f2410a\") " pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:07.314145 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.314114 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dfe559be-dba3-40b1-a187-6bb419f2410a-must-gather-output\") pod \"must-gather-svz8j\" (UID: \"dfe559be-dba3-40b1-a187-6bb419f2410a\") " pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:07.314311 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.314155 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnbdq\" (UniqueName: \"kubernetes.io/projected/dfe559be-dba3-40b1-a187-6bb419f2410a-kube-api-access-mnbdq\") pod \"must-gather-svz8j\" (UID: \"dfe559be-dba3-40b1-a187-6bb419f2410a\") " pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:07.314568 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.314547 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dfe559be-dba3-40b1-a187-6bb419f2410a-must-gather-output\") pod \"must-gather-svz8j\" (UID: \"dfe559be-dba3-40b1-a187-6bb419f2410a\") " pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:07.323643 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.323614 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnbdq\" (UniqueName: \"kubernetes.io/projected/dfe559be-dba3-40b1-a187-6bb419f2410a-kube-api-access-mnbdq\") pod \"must-gather-svz8j\" (UID: \"dfe559be-dba3-40b1-a187-6bb419f2410a\") " pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:07.511487 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.511450 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:07.643598 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.643564 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-brn9x/must-gather-svz8j"] Aug 01 00:12:07.647001 ip-10-0-131-119 kubenswrapper[2582]: W0801 00:12:07.646974 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfe559be_dba3_40b1_a187_6bb419f2410a.slice/crio-90dbac3d7b5b896e700353253d970b7683f5b353ed083f7bf594305abe914185 WatchSource:0}: Error finding container 90dbac3d7b5b896e700353253d970b7683f5b353ed083f7bf594305abe914185: Status 404 returned error can't find the container with id 90dbac3d7b5b896e700353253d970b7683f5b353ed083f7bf594305abe914185 Aug 01 00:12:07.648858 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:07.648841 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 01 00:12:08.300297 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:08.300265 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-brn9x/must-gather-svz8j" event={"ID":"dfe559be-dba3-40b1-a187-6bb419f2410a","Type":"ContainerStarted","Data":"90dbac3d7b5b896e700353253d970b7683f5b353ed083f7bf594305abe914185"} Aug 01 00:12:14.329085 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:14.328991 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-brn9x/must-gather-svz8j" event={"ID":"dfe559be-dba3-40b1-a187-6bb419f2410a","Type":"ContainerStarted","Data":"2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c"} Aug 01 00:12:14.329085 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:14.329037 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-brn9x/must-gather-svz8j" event={"ID":"dfe559be-dba3-40b1-a187-6bb419f2410a","Type":"ContainerStarted","Data":"e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355"} Aug 01 00:12:14.345647 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:14.345593 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-brn9x/must-gather-svz8j" podStartSLOduration=1.208837268 podStartE2EDuration="7.345576894s" totalImagesPullingTime="6.136739626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:12:14.344802367 +0000 UTC m=+1551.504651480" watchObservedRunningTime="2026-08-01 00:12:14.345576894 +0000 UTC m=+1551.505426006" Aug 01 00:12:37.421955 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:37.421920 2582 generic.go:356] "Generic (PLEG): container finished" podID="dfe559be-dba3-40b1-a187-6bb419f2410a" containerID="e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355" exitCode=0 Aug 01 00:12:37.422394 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:37.421991 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-brn9x/must-gather-svz8j" event={"ID":"dfe559be-dba3-40b1-a187-6bb419f2410a","Type":"ContainerDied","Data":"e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355"} Aug 01 00:12:37.422394 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:37.422321 2582 scope.go:117] "RemoveContainer" containerID="e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355" Aug 01 00:12:37.928869 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:37.928842 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-brn9x_must-gather-svz8j_dfe559be-dba3-40b1-a187-6bb419f2410a/gather/0.log" Aug 01 00:12:41.891948 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:41.891913 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-l9sh5_03fa2c21-f2f9-450a-ac02-5a0de6eef3ee/global-pull-secret-syncer/0.log" Aug 01 00:12:41.968240 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:41.968207 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z6tfz_c911e907-1c9e-4f5a-b131-5586c281a7ed/konnectivity-agent/0.log" Aug 01 00:12:42.018672 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:42.018647 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-119.ec2.internal_148fdac59207384bb5b95d23db64dc63/haproxy/0.log" Aug 01 00:12:43.345885 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:43.345853 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-brn9x/must-gather-svz8j"] Aug 01 00:12:43.346458 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:43.346049 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-brn9x/must-gather-svz8j" podUID="dfe559be-dba3-40b1-a187-6bb419f2410a" containerName="copy" containerID="cri-o://2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c" gracePeriod=2 Aug 01 00:12:43.348899 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:12:43.348863 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-svz8j\" is forbidden: User \"system:node:ip-10-0-131-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-brn9x\": no relationship found between node 'ip-10-0-131-119.ec2.internal' and this object" podUID="dfe559be-dba3-40b1-a187-6bb419f2410a" pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:43.350153 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:43.349807 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-brn9x/must-gather-svz8j"] Aug 01 00:12:43.370275 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:12:43.370235 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-svz8j\" is forbidden: User \"system:node:ip-10-0-131-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-brn9x\": no relationship found between node 'ip-10-0-131-119.ec2.internal' and this object" podUID="dfe559be-dba3-40b1-a187-6bb419f2410a" pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:43.575710 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:43.575686 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-brn9x_must-gather-svz8j_dfe559be-dba3-40b1-a187-6bb419f2410a/copy/0.log" Aug 01 00:12:43.577201 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:43.577178 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:43.734893 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:43.734863 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dfe559be-dba3-40b1-a187-6bb419f2410a-must-gather-output\") pod \"dfe559be-dba3-40b1-a187-6bb419f2410a\" (UID: \"dfe559be-dba3-40b1-a187-6bb419f2410a\") " Aug 01 00:12:43.735084 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:43.734918 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnbdq\" (UniqueName: \"kubernetes.io/projected/dfe559be-dba3-40b1-a187-6bb419f2410a-kube-api-access-mnbdq\") pod \"dfe559be-dba3-40b1-a187-6bb419f2410a\" (UID: \"dfe559be-dba3-40b1-a187-6bb419f2410a\") " Aug 01 00:12:43.736993 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:43.736961 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfe559be-dba3-40b1-a187-6bb419f2410a-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "dfe559be-dba3-40b1-a187-6bb419f2410a" (UID: "dfe559be-dba3-40b1-a187-6bb419f2410a"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 01 00:12:43.737160 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:43.737138 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfe559be-dba3-40b1-a187-6bb419f2410a-kube-api-access-mnbdq" (OuterVolumeSpecName: "kube-api-access-mnbdq") pod "dfe559be-dba3-40b1-a187-6bb419f2410a" (UID: "dfe559be-dba3-40b1-a187-6bb419f2410a"). InnerVolumeSpecName "kube-api-access-mnbdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 01 00:12:43.836052 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:43.836013 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnbdq\" (UniqueName: \"kubernetes.io/projected/dfe559be-dba3-40b1-a187-6bb419f2410a-kube-api-access-mnbdq\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:12:43.836052 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:43.836043 2582 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dfe559be-dba3-40b1-a187-6bb419f2410a-must-gather-output\") on node \"ip-10-0-131-119.ec2.internal\" DevicePath \"\"" Aug 01 00:12:44.449557 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:44.449524 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-brn9x_must-gather-svz8j_dfe559be-dba3-40b1-a187-6bb419f2410a/copy/0.log" Aug 01 00:12:44.449956 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:44.449895 2582 generic.go:356] "Generic (PLEG): container finished" podID="dfe559be-dba3-40b1-a187-6bb419f2410a" containerID="2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c" exitCode=143 Aug 01 00:12:44.449956 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:44.449950 2582 scope.go:117] "RemoveContainer" containerID="2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c" Aug 01 00:12:44.450078 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:44.449953 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-brn9x/must-gather-svz8j" Aug 01 00:12:44.457819 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:44.457421 2582 scope.go:117] "RemoveContainer" containerID="e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355" Aug 01 00:12:44.469280 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:44.469258 2582 scope.go:117] "RemoveContainer" containerID="2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c" Aug 01 00:12:44.469594 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:12:44.469571 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c\": container with ID starting with 2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c not found: ID does not exist" containerID="2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c" Aug 01 00:12:44.469668 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:44.469603 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c"} err="failed to get container status \"2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c\": rpc error: code = NotFound desc = could not find container \"2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c\": container with ID starting with 2cf2de091d085406c4fe35f3d2768e6685a51d0996930e3554550e9d018dfe1c not found: ID does not exist" Aug 01 00:12:44.469668 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:44.469623 2582 scope.go:117] "RemoveContainer" containerID="e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355" Aug 01 00:12:44.469864 ip-10-0-131-119 kubenswrapper[2582]: E0801 00:12:44.469843 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355\": container with ID starting with e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355 not found: ID does not exist" containerID="e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355" Aug 01 00:12:44.469928 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:44.469877 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355"} err="failed to get container status \"e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355\": rpc error: code = NotFound desc = could not find container \"e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355\": container with ID starting with e77d7463c0096bfcb79e7b83db0f4c1c7a0eaeb3d2f83a6140674dc5d6133355 not found: ID does not exist" Aug 01 00:12:45.374604 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:45.372656 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfe559be-dba3-40b1-a187-6bb419f2410a" path="/var/lib/kubelet/pods/dfe559be-dba3-40b1-a187-6bb419f2410a/volumes" Aug 01 00:12:46.288863 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.288831 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-rlmd9_8c28b380-c1ce-4d77-9942-564a8ad8dbc5/kube-state-metrics/0.log" Aug 01 00:12:46.307606 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.307579 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-rlmd9_8c28b380-c1ce-4d77-9942-564a8ad8dbc5/kube-rbac-proxy-main/0.log" Aug 01 00:12:46.332759 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.332729 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-rlmd9_8c28b380-c1ce-4d77-9942-564a8ad8dbc5/kube-rbac-proxy-self/0.log" Aug 01 00:12:46.385068 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.385041 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-xwmhj_b2cd92d2-0334-4923-b45d-35da209bdc80/monitoring-plugin/0.log" Aug 01 00:12:46.415139 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.415110 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6gc4v_61d9f4f9-7224-4b47-8e7c-e939af9bc010/node-exporter/0.log" Aug 01 00:12:46.441153 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.441126 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6gc4v_61d9f4f9-7224-4b47-8e7c-e939af9bc010/kube-rbac-proxy/0.log" Aug 01 00:12:46.459689 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.459627 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6gc4v_61d9f4f9-7224-4b47-8e7c-e939af9bc010/init-textfile/0.log" Aug 01 00:12:46.695286 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.695258 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-zq5c9_7eaf76d4-9339-44fc-9969-5b7e95509dbd/kube-rbac-proxy-main/0.log" Aug 01 00:12:46.716380 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.716307 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-zq5c9_7eaf76d4-9339-44fc-9969-5b7e95509dbd/kube-rbac-proxy-self/0.log" Aug 01 00:12:46.737235 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.737202 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-zq5c9_7eaf76d4-9339-44fc-9969-5b7e95509dbd/openshift-state-metrics/0.log" Aug 01 00:12:46.775324 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.775297 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_669a0ac6-317a-46c8-89e8-56c0291a9b7f/prometheus/0.log" Aug 01 00:12:46.792940 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.792915 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_669a0ac6-317a-46c8-89e8-56c0291a9b7f/config-reloader/0.log" Aug 01 00:12:46.811955 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.811927 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_669a0ac6-317a-46c8-89e8-56c0291a9b7f/thanos-sidecar/0.log" Aug 01 00:12:46.831588 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.831558 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_669a0ac6-317a-46c8-89e8-56c0291a9b7f/kube-rbac-proxy-web/0.log" Aug 01 00:12:46.851001 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.850979 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_669a0ac6-317a-46c8-89e8-56c0291a9b7f/kube-rbac-proxy/0.log" Aug 01 00:12:46.871579 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.871552 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_669a0ac6-317a-46c8-89e8-56c0291a9b7f/kube-rbac-proxy-thanos/0.log" Aug 01 00:12:46.897004 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:46.896977 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_669a0ac6-317a-46c8-89e8-56c0291a9b7f/init-config-reloader/0.log" Aug 01 00:12:49.426214 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.426181 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-6qbq8_8b00e295-a506-43f7-b005-3de7ed37ea94/download-server/0.log" Aug 01 00:12:49.985750 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.985714 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml"] Aug 01 00:12:49.985998 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.985987 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfe559be-dba3-40b1-a187-6bb419f2410a" containerName="gather" Aug 01 00:12:49.986056 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.986000 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfe559be-dba3-40b1-a187-6bb419f2410a" containerName="gather" Aug 01 00:12:49.986056 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.986013 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfe559be-dba3-40b1-a187-6bb419f2410a" containerName="copy" Aug 01 00:12:49.986056 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.986018 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfe559be-dba3-40b1-a187-6bb419f2410a" containerName="copy" Aug 01 00:12:49.986146 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.986074 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfe559be-dba3-40b1-a187-6bb419f2410a" containerName="copy" Aug 01 00:12:49.986146 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.986083 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfe559be-dba3-40b1-a187-6bb419f2410a" containerName="gather" Aug 01 00:12:49.990029 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.990007 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:49.992633 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.992612 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xgqkp\"/\"openshift-service-ca.crt\"" Aug 01 00:12:49.992720 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.992705 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xgqkp\"/\"kube-root-ca.crt\"" Aug 01 00:12:49.993635 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.993620 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xgqkp\"/\"default-dockercfg-kblfc\"" Aug 01 00:12:49.997715 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:49.997694 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml"] Aug 01 00:12:50.085848 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.085817 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-podres\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.086011 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.085862 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-sys\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.086011 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.085898 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-proc\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.086011 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.085949 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fjn5\" (UniqueName: \"kubernetes.io/projected/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-kube-api-access-5fjn5\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.086011 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.086007 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-lib-modules\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.186508 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.186472 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-sys\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.186688 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.186516 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-proc\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.186688 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.186539 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fjn5\" (UniqueName: \"kubernetes.io/projected/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-kube-api-access-5fjn5\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.186688 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.186577 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-lib-modules\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.186688 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.186598 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-sys\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.186688 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.186609 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-proc\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.186688 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.186614 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-podres\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.186964 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.186699 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-podres\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.186964 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.186734 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-lib-modules\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.194557 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.194532 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fjn5\" (UniqueName: \"kubernetes.io/projected/18b0e7c1-6ec8-4c14-bc73-50f46f53dba0-kube-api-access-5fjn5\") pod \"perf-node-gather-daemonset-vpxml\" (UID: \"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.299751 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.299654 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:50.421087 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.421054 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml"] Aug 01 00:12:50.423873 ip-10-0-131-119 kubenswrapper[2582]: W0801 00:12:50.423845 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod18b0e7c1_6ec8_4c14_bc73_50f46f53dba0.slice/crio-71446d5521cbaff4848d09b88626da52ae30b17ce6dec9dc6925b59029cfe413 WatchSource:0}: Error finding container 71446d5521cbaff4848d09b88626da52ae30b17ce6dec9dc6925b59029cfe413: Status 404 returned error can't find the container with id 71446d5521cbaff4848d09b88626da52ae30b17ce6dec9dc6925b59029cfe413 Aug 01 00:12:50.472652 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.472625 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" event={"ID":"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0","Type":"ContainerStarted","Data":"71446d5521cbaff4848d09b88626da52ae30b17ce6dec9dc6925b59029cfe413"} Aug 01 00:12:50.864821 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.864795 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-frqxm_1481c3cd-a8e1-4d75-a73a-f675c5b936a0/dns/0.log" Aug 01 00:12:50.883447 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.883424 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-frqxm_1481c3cd-a8e1-4d75-a73a-f675c5b936a0/kube-rbac-proxy/0.log" Aug 01 00:12:50.949672 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:50.949646 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6czd7_16ff1b3d-8225-4e5e-a2df-3ea9b0205971/dns-node-resolver/0.log" Aug 01 00:12:51.456731 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:51.456703 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-659bffd457-92tkj_065b662e-d891-4fd5-a2a9-b610d1aa991a/registry/0.log" Aug 01 00:12:51.477540 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:51.477504 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" event={"ID":"18b0e7c1-6ec8-4c14-bc73-50f46f53dba0","Type":"ContainerStarted","Data":"0a7698dec91082fa819eea7be50cd3785dca76e83c0da12a740b2111f104dd20"} Aug 01 00:12:51.477871 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:51.477628 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:51.493183 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:51.493139 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" podStartSLOduration=2.493124076 podStartE2EDuration="2.493124076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-01 00:12:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:12:51.492074821 +0000 UTC m=+1588.651923932" watchObservedRunningTime="2026-08-01 00:12:51.493124076 +0000 UTC m=+1588.652973189" Aug 01 00:12:51.518523 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:51.518502 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qwln9_2262118f-b73c-4dbf-9ca6-d14c05495100/node-ca/0.log" Aug 01 00:12:52.590997 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:52.590970 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jz6nv_16a41c32-9c4a-4ea7-9c1a-9aeb6f9b33cf/serve-healthcheck-canary/0.log" Aug 01 00:12:52.930616 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:52.930541 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5xm8p_053b510c-cca6-4383-84b2-836a9d686abc/kube-rbac-proxy/0.log" Aug 01 00:12:52.948594 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:52.948557 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5xm8p_053b510c-cca6-4383-84b2-836a9d686abc/exporter/0.log" Aug 01 00:12:52.967627 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:52.967600 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5xm8p_053b510c-cca6-4383-84b2-836a9d686abc/extractor/0.log" Aug 01 00:12:55.481723 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:55.481695 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-qhnrw_0277ebea-21b6-4021-b285-cee18ac86e28/s3-init/0.log" Aug 01 00:12:55.498086 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:55.498049 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-qhnrw_0277ebea-21b6-4021-b285-cee18ac86e28/download-hf-model/0.log" Aug 01 00:12:57.493324 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:57.493297 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-vpxml" Aug 01 00:12:59.075935 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:59.075906 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-47cbn_85de7af4-868a-4913-983b-5691cc557068/migrator/0.log" Aug 01 00:12:59.094800 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:12:59.094774 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-47cbn_85de7af4-868a-4913-983b-5691cc557068/graceful-termination/0.log" Aug 01 00:13:00.818238 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:00.818163 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smx29_76114bf1-387d-413b-8b61-9cd4d084d3d9/kube-multus-additional-cni-plugins/0.log" Aug 01 00:13:00.843709 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:00.843679 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smx29_76114bf1-387d-413b-8b61-9cd4d084d3d9/egress-router-binary-copy/0.log" Aug 01 00:13:00.863463 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:00.863431 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smx29_76114bf1-387d-413b-8b61-9cd4d084d3d9/cni-plugins/0.log" Aug 01 00:13:00.883658 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:00.883633 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smx29_76114bf1-387d-413b-8b61-9cd4d084d3d9/bond-cni-plugin/0.log" Aug 01 00:13:00.907651 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:00.907622 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smx29_76114bf1-387d-413b-8b61-9cd4d084d3d9/routeoverride-cni/0.log" Aug 01 00:13:00.929712 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:00.929685 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smx29_76114bf1-387d-413b-8b61-9cd4d084d3d9/whereabouts-cni-bincopy/0.log" Aug 01 00:13:00.951961 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:00.951927 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smx29_76114bf1-387d-413b-8b61-9cd4d084d3d9/whereabouts-cni/0.log" Aug 01 00:13:01.035354 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:01.035326 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tmzjt_f03a7f95-a27b-408a-afad-c7f07ac5bb58/kube-multus/0.log" Aug 01 00:13:01.055649 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:01.055625 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dbgxq_9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199/network-metrics-daemon/0.log" Aug 01 00:13:01.073656 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:01.073594 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dbgxq_9584b452-2eaa-4ff8-8ee3-ab4ed6fe4199/kube-rbac-proxy/0.log" Aug 01 00:13:02.380816 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:02.380727 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jr47x_80ad104b-689c-49ca-ad79-0a682d464661/ovn-controller/0.log" Aug 01 00:13:02.413031 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:02.413006 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jr47x_80ad104b-689c-49ca-ad79-0a682d464661/ovn-acl-logging/0.log" Aug 01 00:13:02.451045 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:02.451018 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jr47x_80ad104b-689c-49ca-ad79-0a682d464661/kube-rbac-proxy-node/0.log" Aug 01 00:13:02.486293 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:02.486268 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jr47x_80ad104b-689c-49ca-ad79-0a682d464661/kube-rbac-proxy-ovn-metrics/0.log" Aug 01 00:13:02.513626 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:02.513597 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jr47x_80ad104b-689c-49ca-ad79-0a682d464661/northd/0.log" Aug 01 00:13:02.552262 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:02.552230 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jr47x_80ad104b-689c-49ca-ad79-0a682d464661/nbdb/0.log" Aug 01 00:13:02.575313 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:02.575285 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jr47x_80ad104b-689c-49ca-ad79-0a682d464661/sbdb/0.log" Aug 01 00:13:02.615142 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:02.615113 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jr47x_80ad104b-689c-49ca-ad79-0a682d464661/ovnkube-controller/0.log" Aug 01 00:13:03.600312 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:03.600285 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-wlkds_49904e8b-d888-442a-bab8-1e3c3f83d2e7/network-check-target-container/0.log" Aug 01 00:13:04.521501 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:04.521462 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-8mddg_2981911a-31f1-4d77-ab06-d45688555c73/iptables-alerter/0.log" Aug 01 00:13:05.221509 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:05.221483 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-g6zzk_4a029017-22f0-47b7-aaef-ce8d7b3777ed/tuned/0.log" Aug 01 00:13:08.136325 ip-10-0-131-119 kubenswrapper[2582]: I0801 00:13:08.136297 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-98c88fbb5-9ns9t_b30515f9-283a-403d-b427-6680bf93c45b/service-ca-controller/0.log"