Jul 31 19:23:24.615625 ip-10-0-136-28 systemd[1]: Starting Kubernetes Kubelet... Jul 31 19:23:24.995256 ip-10-0-136-28 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 19:23:24.995256 ip-10-0-136-28 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 31 19:23:24.995256 ip-10-0-136-28 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 19:23:24.995256 ip-10-0-136-28 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 31 19:23:24.995256 ip-10-0-136-28 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 19:23:24.996008 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:24.995249 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 31 19:23:24.999681 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999661 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 19:23:24.999681 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999680 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 19:23:24.999681 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999686 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999691 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999697 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999702 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999707 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999711 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999715 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999718 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999722 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999727 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999731 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999735 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999739 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999743 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999746 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999750 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999753 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999757 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999761 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999765 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 19:23:24.999887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999768 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999772 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999776 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999780 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999785 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999789 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999793 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999796 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999800 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999804 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999808 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999812 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999816 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999820 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999825 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999829 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999834 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999839 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999843 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999847 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 19:23:25.000697 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999851 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999855 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999859 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999863 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999867 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999871 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999875 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999879 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999883 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999886 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999890 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999894 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999898 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999902 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999906 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999910 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999914 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999919 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999923 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999927 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 19:23:25.001365 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999931 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999938 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999945 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999949 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999953 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999957 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999962 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999967 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999971 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999975 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999979 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999983 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999987 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999991 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999995 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:24.999999 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000003 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000007 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000011 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 19:23:25.001864 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000015 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000019 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000023 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000027 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000031 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000035 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000039 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000043 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000047 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000051 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000055 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000059 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000063 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000067 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000071 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000075 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000080 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 19:23:25.002311 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.000085 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002617 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002629 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002634 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002638 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002642 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002646 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002651 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002654 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002658 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002662 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002666 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002670 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002674 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002678 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002681 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002686 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002691 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002697 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002701 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 19:23:25.003024 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002705 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002709 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002713 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002717 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002721 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002726 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002729 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002734 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002737 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002742 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002746 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002750 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002757 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002761 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002766 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002770 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002774 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002778 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002782 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002786 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 19:23:25.003834 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002790 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002794 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002798 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002802 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002806 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002810 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002814 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002818 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002822 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002826 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002830 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002835 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002839 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002842 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002846 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002850 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002854 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002857 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002862 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002868 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 19:23:25.004734 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002872 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002877 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002882 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002887 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002891 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002898 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002904 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002909 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002914 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002919 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002923 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002928 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002931 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002935 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002939 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002943 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002947 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002951 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002955 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 19:23:25.005610 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002959 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002963 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002967 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002974 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002978 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002982 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002986 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002990 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002994 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.002998 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003002 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003007 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003013 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003019 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003023 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003027 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003031 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003036 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003040 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003045 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 19:23:25.006077 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003049 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003692 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003701 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003705 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003709 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003713 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003718 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003722 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003727 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003731 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003735 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003739 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003743 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003747 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003752 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003756 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003760 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003764 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003768 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003772 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 19:23:25.006653 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003776 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003779 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003784 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003788 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003792 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003796 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003800 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003805 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003809 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003813 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003820 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003826 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003831 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003835 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003840 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003844 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003848 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003852 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003857 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 19:23:25.007356 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003862 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003866 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003870 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003874 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003878 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003886 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003891 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003895 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003900 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003905 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003913 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003917 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003922 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003926 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003930 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003934 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003938 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003942 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003946 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 19:23:25.007903 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003950 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003954 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003958 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003962 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003965 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003970 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003975 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003979 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003983 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003987 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003991 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.003995 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004000 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004006 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004011 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004015 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004019 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004023 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004027 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004031 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 19:23:25.008368 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004035 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004039 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004043 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004047 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004051 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004055 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004059 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004063 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004069 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004073 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004076 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004080 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004085 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004090 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004094 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004098 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004101 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004106 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004110 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004114 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 19:23:25.008912 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004118 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.004122 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004218 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004228 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004237 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004244 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004251 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004255 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004262 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004268 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004273 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004278 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004284 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004289 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004294 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004298 2565 flags.go:64] FLAG: --cgroup-root="" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004302 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004307 2565 flags.go:64] FLAG: --client-ca-file="" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004311 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004317 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004324 2565 flags.go:64] FLAG: --cluster-domain="" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004329 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004333 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004356 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 31 19:23:25.009456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004361 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004367 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004371 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004376 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004381 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004386 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004391 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004395 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004401 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004406 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004412 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004416 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004421 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004425 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004430 2565 flags.go:64] FLAG: --enable-server="true" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004435 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004442 2565 flags.go:64] FLAG: --event-burst="100" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004447 2565 flags.go:64] FLAG: --event-qps="50" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004451 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004455 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004460 2565 flags.go:64] FLAG: --eviction-hard="" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004466 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004471 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004475 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004480 2565 flags.go:64] FLAG: --eviction-soft="" Jul 31 19:23:25.010004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004485 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004490 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004494 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004498 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004503 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004508 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004512 2565 flags.go:64] FLAG: --feature-gates="" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004518 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004523 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004527 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004533 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004537 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004542 2565 flags.go:64] FLAG: --help="false" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004547 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004552 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004557 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004562 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004568 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004573 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004577 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004582 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004587 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004591 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 31 19:23:25.010622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004596 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004601 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004605 2565 flags.go:64] FLAG: --kube-reserved="" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004610 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004614 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004619 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004623 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004627 2565 flags.go:64] FLAG: --lock-file="" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004631 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004636 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004640 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004649 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004653 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004657 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004662 2565 flags.go:64] FLAG: --logging-format="text" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004666 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004671 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004675 2565 flags.go:64] FLAG: --manifest-url="" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004680 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004687 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004691 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004697 2565 flags.go:64] FLAG: --max-pods="110" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004702 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004707 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004711 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 31 19:23:25.011182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004716 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004721 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004726 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004730 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004743 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004747 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004752 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004756 2565 flags.go:64] FLAG: --pod-cidr="" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004760 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004768 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004773 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004778 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004782 2565 flags.go:64] FLAG: --port="10250" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004787 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004792 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-069b51e9cda95c810" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004796 2565 flags.go:64] FLAG: --qos-reserved="" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004801 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004805 2565 flags.go:64] FLAG: --register-node="true" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004810 2565 flags.go:64] FLAG: --register-with-taints="" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004816 2565 flags.go:64] FLAG: --registry-burst="10" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004820 2565 flags.go:64] FLAG: --registry-qps="5" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004824 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004828 2565 flags.go:64] FLAG: --reserved-memory="" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004834 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004838 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 31 19:23:25.011854 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004843 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004847 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004852 2565 flags.go:64] FLAG: --runonce="false" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004856 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004864 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004868 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004873 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004877 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004882 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004887 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004892 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004899 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004903 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004908 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004912 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004917 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004921 2565 flags.go:64] FLAG: --system-cgroups="" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004925 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004934 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004939 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004943 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004950 2565 flags.go:64] FLAG: --tls-min-version="" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004954 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004959 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004963 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 31 19:23:25.012458 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004968 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004974 2565 flags.go:64] FLAG: --v="2" Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004980 2565 flags.go:64] FLAG: --version="false" Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004986 2565 flags.go:64] FLAG: --vmodule="" Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.004997 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.005002 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005173 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005182 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005187 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005192 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005197 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005203 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005208 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005212 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005217 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005221 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005225 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005230 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005238 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005243 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 19:23:25.013079 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005247 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005252 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005256 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005260 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005265 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005269 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005274 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005278 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005282 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005286 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005290 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005294 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005298 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005303 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005306 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005310 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005314 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005318 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005331 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 19:23:25.013867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005355 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005360 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005364 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005368 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005374 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005378 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005382 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005386 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005390 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005394 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005399 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005405 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005409 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005413 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005417 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005421 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005425 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005429 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005433 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005437 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 19:23:25.014491 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005440 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005444 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005448 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005452 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005456 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005460 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005464 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005468 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005472 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005476 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005480 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005484 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005488 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005491 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005496 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005500 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005505 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005510 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005514 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005518 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 19:23:25.015011 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005522 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005526 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005530 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005535 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005540 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005545 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005548 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005553 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005557 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005561 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005565 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005569 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005573 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005577 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005581 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005585 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005590 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005593 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005597 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005601 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 31 19:23:25.015501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005605 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005609 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005613 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005616 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005620 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.005623 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.006122 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.013880 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.013909 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.013934 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.013947 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014278 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014287 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014290 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 19:23:25.015963 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014293 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014296 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014298 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014302 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014304 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014308 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014310 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014313 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014316 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014318 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014321 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014324 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014326 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014329 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014331 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014334 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014336 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014353 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014355 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 19:23:25.016402 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014358 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014360 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014363 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014366 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014368 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014370 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014373 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014379 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014382 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014385 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014389 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014394 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014396 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014399 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014401 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014404 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014407 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014409 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014412 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014415 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 19:23:25.016847 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014417 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014420 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014423 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014427 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014430 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014433 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014435 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014438 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014440 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014443 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014445 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014448 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014451 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014453 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014456 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014458 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014461 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014463 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014466 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 19:23:25.017382 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014469 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014473 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014476 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014479 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014482 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014485 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014487 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014490 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014493 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014496 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014499 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014502 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014504 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014507 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014510 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014512 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014515 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014517 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014525 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014528 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 19:23:25.017833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014530 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014533 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014535 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014538 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014540 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014542 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014545 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014547 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014550 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014552 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014554 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014557 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014559 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014562 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014564 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014567 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014569 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 19:23:25.018336 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014571 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.014576 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014695 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014699 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014702 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014705 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014707 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014709 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014712 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014714 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014717 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014719 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014721 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014724 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014732 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 19:23:25.018830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014735 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014739 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014742 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014744 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014747 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014749 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014751 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014754 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014756 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014759 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014761 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014763 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014766 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014768 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014771 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014773 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014775 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014778 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014780 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014782 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 19:23:25.019183 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014785 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014787 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014789 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014792 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014794 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014796 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014799 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014801 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014803 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014806 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014809 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014813 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014820 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014823 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014825 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014828 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014830 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014833 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014835 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 19:23:25.019675 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014838 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014841 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014843 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014845 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014848 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014850 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014852 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014855 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014857 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014860 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014862 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014864 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014867 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014869 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014871 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014874 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014876 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014878 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014881 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 19:23:25.020120 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014883 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014885 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014888 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014890 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014892 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014895 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014897 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014904 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014906 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014908 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014911 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014914 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014916 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014919 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014921 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014923 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014925 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014928 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014931 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014936 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 19:23:25.020584 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014938 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014941 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014944 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014946 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014949 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014951 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014954 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:25.014956 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.014961 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.015068 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.017002 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.018307 2565 server.go:1019] "Starting client certificate rotation" Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.018375 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 31 19:23:25.021046 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.018416 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 31 19:23:25.021379 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.020671 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 31 19:23:25.021987 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.021971 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 31 19:23:25.032880 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.032863 2565 log.go:25] "Validated CRI v1 runtime API" Jul 31 19:23:25.034706 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.034690 2565 log.go:25] "Validated CRI v1 image API" Jul 31 19:23:25.036655 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.036636 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 31 19:23:25.039659 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.039640 2565 fs.go:135] Filesystem UUIDs: map[0cd69e61-d1ee-4d16-b713-ba90a116368e:/dev/nvme0n1p3 5369bc35-da61-4a08-8eff-38ba38ded396:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 31 19:23:25.039738 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.039657 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 31 19:23:25.044382 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.044361 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 31 19:23:25.045690 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.045567 2565 manager.go:216] Machine: {Timestamp:2026-07-31 19:23:25.043691872 +0000 UTC m=+0.315039217 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3196728 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec278a8726a096296ac8557ff217b486 SystemUUID:ec278a87-26a0-9629-6ac8-557ff217b486 BootID:e92603bd-3eb7-40c2-8eae-573a1408383b Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:35:c6:4b:b3:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:35:c6:4b:b3:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:72:6f:9b:96:91 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 31 19:23:25.045690 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.045685 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 31 19:23:25.045794 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.045762 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 31 19:23:25.050031 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.050006 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 31 19:23:25.050163 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.050033 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-28.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 31 19:23:25.050708 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.050699 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 31 19:23:25.050744 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.050710 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 31 19:23:25.050744 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.050717 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 31 19:23:25.051619 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.051610 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 31 19:23:25.052305 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.052296 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 31 19:23:25.057088 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.057073 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 31 19:23:25.057206 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.057196 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 31 19:23:25.057247 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.057227 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 31 19:23:25.057247 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.057237 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 31 19:23:25.057313 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.057253 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 31 19:23:25.057313 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.057262 2565 kubelet.go:398] "Adding apiserver pod source" Jul 31 19:23:25.057313 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.057271 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 31 19:23:25.057676 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.057665 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 31 19:23:25.057715 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.057690 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 31 19:23:25.059712 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.059691 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 31 19:23:25.060927 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.060914 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 31 19:23:25.060974 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.060944 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 31 19:23:25.061852 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061841 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 31 19:23:25.061887 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061858 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 31 19:23:25.061887 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061865 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 31 19:23:25.061887 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061872 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 31 19:23:25.061887 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061882 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 31 19:23:25.061887 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061888 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 31 19:23:25.062016 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061898 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 31 19:23:25.062016 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061905 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 31 19:23:25.062016 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061912 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 31 19:23:25.062016 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061919 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 31 19:23:25.062016 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061929 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 31 19:23:25.062016 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.061935 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 31 19:23:25.062232 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.062219 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lrf8p" Jul 31 19:23:25.062623 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.062612 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 31 19:23:25.062656 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.062626 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 31 19:23:25.065243 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.065223 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-28.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 31 19:23:25.065836 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.065816 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 31 19:23:25.065879 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.065848 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 31 19:23:25.066230 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.066214 2565 server.go:1268] "Started kubelet" Jul 31 19:23:25.066281 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.066253 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 31 19:23:25.066324 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.066279 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 31 19:23:25.066382 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.066367 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 31 19:23:25.067152 ip-10-0-136-28 systemd[1]: Started Kubernetes Kubelet. Jul 31 19:23:25.068031 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.068010 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 31 19:23:25.068177 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.068165 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 31 19:23:25.069262 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.069247 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lrf8p" Jul 31 19:23:25.072162 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.072129 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 31 19:23:25.073181 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.073167 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 31 19:23:25.073249 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.073166 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 31 19:23:25.073403 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.073380 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:25.073950 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.073814 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 31 19:23:25.073950 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.073835 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 31 19:23:25.073950 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.073948 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 31 19:23:25.074098 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.073962 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 31 19:23:25.074098 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.073986 2565 factory.go:55] Registering systemd factory Jul 31 19:23:25.074098 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.073998 2565 factory.go:223] Registration of the systemd container factory successfully Jul 31 19:23:25.074098 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.073999 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 31 19:23:25.074302 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.074214 2565 factory.go:153] Registering CRI-O factory Jul 31 19:23:25.074302 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.074227 2565 factory.go:223] Registration of the crio container factory successfully Jul 31 19:23:25.074302 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.074276 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 31 19:23:25.074302 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.074300 2565 factory.go:103] Registering Raw factory Jul 31 19:23:25.074494 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.074314 2565 manager.go:1196] Started watching for new ooms in manager Jul 31 19:23:25.075683 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.075664 2565 manager.go:349] Starting recovery of all containers Jul 31 19:23:25.075993 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.075978 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 19:23:25.078665 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.078623 2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-28.ec2.internal\" not found" node="ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.090566 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.090541 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 31 19:23:25.092026 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.092014 2565 manager.go:354] Recovery completed Jul 31 19:23:25.096455 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.096441 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 19:23:25.098268 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.098252 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientMemory" Jul 31 19:23:25.098354 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.098279 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 19:23:25.098354 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.098289 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientPID" Jul 31 19:23:25.098793 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.098781 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 19:23:25.098864 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.098780 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 31 19:23:25.098864 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.098811 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 31 19:23:25.098864 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.098831 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 31 19:23:25.099444 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.099431 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientMemory" Jul 31 19:23:25.099495 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.099457 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 19:23:25.099495 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.099470 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientPID" Jul 31 19:23:25.100928 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.100914 2565 policy_none.go:49] "None policy: Start" Jul 31 19:23:25.100988 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.100932 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 31 19:23:25.100988 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.100945 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 31 19:23:25.102940 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.102927 2565 policy_none.go:47] "Start" Jul 31 19:23:25.134010 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.133859 2565 manager.go:337] "Starting Device Plugin manager" Jul 31 19:23:25.134093 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.134026 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 31 19:23:25.134093 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.134038 2565 server.go:87] "Starting device plugin registration server" Jul 31 19:23:25.134242 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.134231 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 31 19:23:25.134270 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.134245 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 31 19:23:25.134397 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.134375 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 31 19:23:25.134480 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.134459 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 31 19:23:25.134480 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.134470 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 31 19:23:25.135062 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.135037 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 31 19:23:25.135132 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.135075 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:25.234961 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.234932 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 19:23:25.237081 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.236122 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientMemory" Jul 31 19:23:25.237081 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.236153 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 19:23:25.237081 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.236163 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientPID" Jul 31 19:23:25.237081 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.236188 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.244866 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.244850 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.244918 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.244872 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-28.ec2.internal\": node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:25.257558 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.257508 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:25.257558 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.257516 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 31 19:23:25.257558 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.257549 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 31 19:23:25.257704 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.257567 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 31 19:23:25.257704 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.257600 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 31 19:23:25.259873 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.259853 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 19:23:25.357988 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.357952 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-28.ec2.internal"] Jul 31 19:23:25.358139 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.358036 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 19:23:25.358139 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.357970 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:25.358911 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.358895 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientMemory" Jul 31 19:23:25.358986 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.358924 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 19:23:25.358986 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.358933 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientPID" Jul 31 19:23:25.361166 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.361153 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 19:23:25.361308 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.361293 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.361403 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.361335 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 19:23:25.361820 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.361805 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientMemory" Jul 31 19:23:25.361900 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.361839 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 19:23:25.361900 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.361860 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientPID" Jul 31 19:23:25.361900 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.361868 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientMemory" Jul 31 19:23:25.361900 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.361892 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 19:23:25.362075 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.361906 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientPID" Jul 31 19:23:25.364055 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.364040 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.364125 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.364072 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 19:23:25.364703 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.364686 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientMemory" Jul 31 19:23:25.364766 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.364720 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 19:23:25.364766 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.364736 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeHasSufficientPID" Jul 31 19:23:25.375942 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.375926 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d9b7ef00ef8d5b576fd1871ffcc6c95-config\") pod \"kube-apiserver-proxy-ip-10-0-136-28.ec2.internal\" (UID: \"8d9b7ef00ef8d5b576fd1871ffcc6c95\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.376058 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.375951 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fcbfba3810c2bc625686c61579c9f8d1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal\" (UID: \"fcbfba3810c2bc625686c61579c9f8d1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.376058 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.375969 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcbfba3810c2bc625686c61579c9f8d1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal\" (UID: \"fcbfba3810c2bc625686c61579c9f8d1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.392035 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.392013 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-28.ec2.internal\" not found" node="ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.395326 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.395310 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-28.ec2.internal\" not found" node="ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.459015 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.458984 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:25.476088 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.476064 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fcbfba3810c2bc625686c61579c9f8d1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal\" (UID: \"fcbfba3810c2bc625686c61579c9f8d1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.476199 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.476097 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcbfba3810c2bc625686c61579c9f8d1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal\" (UID: \"fcbfba3810c2bc625686c61579c9f8d1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.476199 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.476123 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d9b7ef00ef8d5b576fd1871ffcc6c95-config\") pod \"kube-apiserver-proxy-ip-10-0-136-28.ec2.internal\" (UID: \"8d9b7ef00ef8d5b576fd1871ffcc6c95\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.476199 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.476159 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fcbfba3810c2bc625686c61579c9f8d1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal\" (UID: \"fcbfba3810c2bc625686c61579c9f8d1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.476199 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.476168 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcbfba3810c2bc625686c61579c9f8d1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal\" (UID: \"fcbfba3810c2bc625686c61579c9f8d1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.476367 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.476229 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d9b7ef00ef8d5b576fd1871ffcc6c95-config\") pod \"kube-apiserver-proxy-ip-10-0-136-28.ec2.internal\" (UID: \"8d9b7ef00ef8d5b576fd1871ffcc6c95\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.559433 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.559379 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:25.659898 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.659855 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:25.694424 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.694399 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.697490 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.697474 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-28.ec2.internal" Jul 31 19:23:25.759987 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.759942 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:25.860444 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.860417 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:25.899021 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:25.898991 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 19:23:25.961036 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:25.961011 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:26.017483 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.017459 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 31 19:23:26.017895 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.017597 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 31 19:23:26.017895 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.017618 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 31 19:23:26.017895 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.017633 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 31 19:23:26.062011 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:26.061985 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:26.072606 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.072571 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-30 19:18:26 +0000 UTC" deadline="2028-02-19 16:26:01.071977669 +0000 UTC" Jul 31 19:23:26.072606 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.072601 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13629h2m34.999380867s" Jul 31 19:23:26.162621 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:26.162563 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:26.255558 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:26.255518 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcbfba3810c2bc625686c61579c9f8d1.slice/crio-3db630b66bb0d3da69b0665bd1453537ca60d77a8a0f5154f7b1d48f15109c21 WatchSource:0}: Error finding container 3db630b66bb0d3da69b0665bd1453537ca60d77a8a0f5154f7b1d48f15109c21: Status 404 returned error can't find the container with id 3db630b66bb0d3da69b0665bd1453537ca60d77a8a0f5154f7b1d48f15109c21 Jul 31 19:23:26.263655 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.262240 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 19:23:26.263655 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.263297 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" event={"ID":"fcbfba3810c2bc625686c61579c9f8d1","Type":"ContainerStarted","Data":"3db630b66bb0d3da69b0665bd1453537ca60d77a8a0f5154f7b1d48f15109c21"} Jul 31 19:23:26.263655 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:26.263486 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:26.369686 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:26.369652 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:26.424593 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:26.424563 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d9b7ef00ef8d5b576fd1871ffcc6c95.slice/crio-9c7b7b2f8e3f989ace08079fea0a61054f3f65f02d6e4d39ab03bafb83999487 WatchSource:0}: Error finding container 9c7b7b2f8e3f989ace08079fea0a61054f3f65f02d6e4d39ab03bafb83999487: Status 404 returned error can't find the container with id 9c7b7b2f8e3f989ace08079fea0a61054f3f65f02d6e4d39ab03bafb83999487 Jul 31 19:23:26.471094 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:26.471055 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-28.ec2.internal\" not found" Jul 31 19:23:26.559247 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.559213 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 19:23:26.574540 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.574518 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" Jul 31 19:23:26.585416 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.585397 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 31 19:23:26.586511 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.586498 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-28.ec2.internal" Jul 31 19:23:26.594426 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:26.594403 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 31 19:23:27.058816 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.058776 2565 apiserver.go:52] "Watching apiserver" Jul 31 19:23:27.065004 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.064977 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 31 19:23:27.065529 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.065506 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69","openshift-image-registry/node-ca-5lpws","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal","openshift-multus/network-metrics-daemon-9qvpw","openshift-network-operator/iptables-alerter-jn4w2","kube-system/global-pull-secret-syncer-xn2mh","openshift-cluster-node-tuning-operator/tuned-btslq","openshift-dns/node-resolver-mkm94","openshift-multus/multus-9762k","openshift-multus/multus-additional-cni-plugins-5pmbt","openshift-network-diagnostics/network-check-target-7rq9h","openshift-ovn-kubernetes/ovnkube-node-d2bmz","kube-system/konnectivity-agent-hpnqm","kube-system/kube-apiserver-proxy-ip-10-0-136-28.ec2.internal"] Jul 31 19:23:27.068099 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.068071 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.070083 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.070060 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 31 19:23:27.070180 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.070138 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hjdqv\"" Jul 31 19:23:27.070180 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.070144 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 31 19:23:27.070764 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.070732 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:27.072677 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.072656 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 31 19:23:27.072773 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.072676 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cxc7n\"" Jul 31 19:23:27.072887 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.072869 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 31 19:23:27.073554 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.073107 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mkm94" Jul 31 19:23:27.073658 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.073640 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 31 19:23:27.074999 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.074979 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 31 19:23:27.075261 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.075245 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 31 19:23:27.075472 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.075456 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sxt8s\"" Jul 31 19:23:27.075649 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.075636 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:27.075716 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.075698 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:27.078856 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.078572 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jn4w2" Jul 31 19:23:27.081085 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.080339 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 31 19:23:27.081085 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.080580 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:27.081085 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.080660 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jmz9v\"" Jul 31 19:23:27.081085 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.080666 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:27.081085 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.080679 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 31 19:23:27.081085 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.080835 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 31 19:23:27.082555 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.082532 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 31 19:23:27.083085 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.083060 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9762k" Jul 31 19:23:27.084732 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.084707 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-modprobe-d\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.084830 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.084744 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-sysctl-conf\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.084830 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.084768 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-run\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.084830 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.084794 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-var-lib-kubelet\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.084830 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.084817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-host\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.085051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.084838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-tuned\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.085051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.084863 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e36226b-6f84-4535-83f8-f3003a0ee2cd-tmp\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.085051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.084889 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-kubernetes\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.085051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.084912 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-sysctl-d\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.085051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.084937 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpm8z\" (UniqueName: \"kubernetes.io/projected/6e36226b-6f84-4535-83f8-f3003a0ee2cd-kube-api-access-hpm8z\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.085051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.084962 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a74545a8-857f-4388-9bd0-4b015d5acaf7-konnectivity-ca\") pod \"konnectivity-agent-hpnqm\" (UID: \"a74545a8-857f-4388-9bd0-4b015d5acaf7\") " pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:27.085051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085012 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9495893a-b14e-4fd9-8999-226d2b5d4eea-hosts-file\") pod \"node-resolver-mkm94\" (UID: \"9495893a-b14e-4fd9-8999-226d2b5d4eea\") " pod="openshift-dns/node-resolver-mkm94" Jul 31 19:23:27.085051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085037 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9495893a-b14e-4fd9-8999-226d2b5d4eea-tmp-dir\") pod \"node-resolver-mkm94\" (UID: \"9495893a-b14e-4fd9-8999-226d2b5d4eea\") " pod="openshift-dns/node-resolver-mkm94" Jul 31 19:23:27.085449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085061 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzgph\" (UniqueName: \"kubernetes.io/projected/9495893a-b14e-4fd9-8999-226d2b5d4eea-kube-api-access-fzgph\") pod \"node-resolver-mkm94\" (UID: \"9495893a-b14e-4fd9-8999-226d2b5d4eea\") " pod="openshift-dns/node-resolver-mkm94" Jul 31 19:23:27.085449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085085 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-sysconfig\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.085449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085110 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-systemd\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.085449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085147 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-sys\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.085449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085170 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-lib-modules\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.085449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085191 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a74545a8-857f-4388-9bd0-4b015d5acaf7-agent-certs\") pod \"konnectivity-agent-hpnqm\" (UID: \"a74545a8-857f-4388-9bd0-4b015d5acaf7\") " pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:27.085449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085221 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 31 19:23:27.085449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085219 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:27.085449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085262 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8f6m\" (UniqueName: \"kubernetes.io/projected/b9991931-cf33-4a99-8185-b3ba855761bc-kube-api-access-f8f6m\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:27.085449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085393 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 31 19:23:27.086099 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085460 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 31 19:23:27.086099 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085676 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 31 19:23:27.086099 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.085718 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2prjx\"" Jul 31 19:23:27.086674 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.086579 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.089330 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.088397 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 31 19:23:27.089330 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.089024 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x2kbn\"" Jul 31 19:23:27.089330 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.089126 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5lpws" Jul 31 19:23:27.089565 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.089494 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 31 19:23:27.089733 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.089710 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 31 19:23:27.094040 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.091715 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 31 19:23:27.094040 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.091776 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 31 19:23:27.094040 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.092122 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5rr89\"" Jul 31 19:23:27.094040 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.092286 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 31 19:23:27.094040 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.093913 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.096270 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.096121 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 31 19:23:27.096270 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.096146 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kt2h6\"" Jul 31 19:23:27.096270 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.096168 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 31 19:23:27.097566 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.097131 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:27.097566 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.097282 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:27.100018 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.099991 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.102506 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.102478 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 31 19:23:27.102609 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.102577 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wwbql\"" Jul 31 19:23:27.102852 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.102834 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 31 19:23:27.102944 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.102907 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 31 19:23:27.103050 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.103033 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 31 19:23:27.103121 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.103091 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 31 19:23:27.103190 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.102838 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 31 19:23:27.104632 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.104602 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 19:23:27.105805 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.105742 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9xtbl" Jul 31 19:23:27.114739 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.114714 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9xtbl" Jul 31 19:23:27.175223 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.175197 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 31 19:23:27.185480 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185448 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-run-k8s-cni-cncf-io\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.185600 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185493 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0650e893-0096-4562-8d1e-aa06dc59efc0-env-overrides\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.185600 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185514 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-socket-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.185600 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185530 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ab3e83f-09c0-4185-ae3b-e5c9b336896f-host-slash\") pod \"iptables-alerter-jn4w2\" (UID: \"4ab3e83f-09c0-4185-ae3b-e5c9b336896f\") " pod="openshift-network-operator/iptables-alerter-jn4w2" Jul 31 19:23:27.185600 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185546 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7sdk\" (UniqueName: \"kubernetes.io/projected/4ab3e83f-09c0-4185-ae3b-e5c9b336896f-kube-api-access-v7sdk\") pod \"iptables-alerter-jn4w2\" (UID: \"4ab3e83f-09c0-4185-ae3b-e5c9b336896f\") " pod="openshift-network-operator/iptables-alerter-jn4w2" Jul 31 19:23:27.185600 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185563 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-os-release\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.185600 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185583 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-run\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.185600 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185600 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-tuned\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.185874 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185614 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e36226b-6f84-4535-83f8-f3003a0ee2cd-tmp\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.185874 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185650 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-multus-conf-dir\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.185874 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185674 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-kubelet\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.185874 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185699 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-etc-openvswitch\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.185874 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185716 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-run-ovn\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.185874 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185736 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-kubernetes\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.185874 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185757 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpm8z\" (UniqueName: \"kubernetes.io/projected/6e36226b-6f84-4535-83f8-f3003a0ee2cd-kube-api-access-hpm8z\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.185874 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185782 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-hostroot\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.185874 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185819 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0650e893-0096-4562-8d1e-aa06dc59efc0-ovn-node-metrics-cert\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.185874 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185845 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5506188d-51d5-4bfa-b204-a8e580d3f1b2-host\") pod \"node-ca-5lpws\" (UID: \"5506188d-51d5-4bfa-b204-a8e580d3f1b2\") " pod="openshift-image-registry/node-ca-5lpws" Jul 31 19:23:27.185874 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185870 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-sysconfig\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185896 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-var-lib-kubelet\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185932 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttdgl\" (UniqueName: \"kubernetes.io/projected/c69f371c-2aba-473f-8df1-01b116efe170-kube-api-access-ttdgl\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185956 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-system-cni-dir\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.185977 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-os-release\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186002 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-node-log\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186018 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-sys\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186034 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42cde026-7512-4487-a633-846e325b55cc-cni-binary-copy\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186062 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-systemd-units\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186094 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8f6m\" (UniqueName: \"kubernetes.io/projected/b9991931-cf33-4a99-8185-b3ba855761bc-kube-api-access-f8f6m\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186110 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-var-lib-kubelet\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186159 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-host\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186181 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a74545a8-857f-4388-9bd0-4b015d5acaf7-konnectivity-ca\") pod \"konnectivity-agent-hpnqm\" (UID: \"a74545a8-857f-4388-9bd0-4b015d5acaf7\") " pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186202 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c69f371c-2aba-473f-8df1-01b116efe170-cni-binary-copy\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.186235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186221 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75srk\" (UniqueName: \"kubernetes.io/projected/5506188d-51d5-4bfa-b204-a8e580d3f1b2-kube-api-access-75srk\") pod \"node-ca-5lpws\" (UID: \"5506188d-51d5-4bfa-b204-a8e580d3f1b2\") " pod="openshift-image-registry/node-ca-5lpws" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186242 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9495893a-b14e-4fd9-8999-226d2b5d4eea-tmp-dir\") pod \"node-resolver-mkm94\" (UID: \"9495893a-b14e-4fd9-8999-226d2b5d4eea\") " pod="openshift-dns/node-resolver-mkm94" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186271 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzgph\" (UniqueName: \"kubernetes.io/projected/9495893a-b14e-4fd9-8999-226d2b5d4eea-kube-api-access-fzgph\") pod \"node-resolver-mkm94\" (UID: \"9495893a-b14e-4fd9-8999-226d2b5d4eea\") " pod="openshift-dns/node-resolver-mkm94" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186287 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-cnibin\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186301 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-run-netns\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186316 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-slash\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186367 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-run-systemd\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186383 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jggvx\" (UniqueName: \"kubernetes.io/projected/42cde026-7512-4487-a633-846e325b55cc-kube-api-access-jggvx\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186405 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-cni-bin\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186443 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0650e893-0096-4562-8d1e-aa06dc59efc0-ovnkube-config\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-systemd\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186479 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a74545a8-857f-4388-9bd0-4b015d5acaf7-agent-certs\") pod \"konnectivity-agent-hpnqm\" (UID: \"a74545a8-857f-4388-9bd0-4b015d5acaf7\") " pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186494 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-log-socket\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186548 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186566 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186607 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-registration-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.186701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186629 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ab3e83f-09c0-4185-ae3b-e5c9b336896f-iptables-alerter-script\") pod \"iptables-alerter-jn4w2\" (UID: \"4ab3e83f-09c0-4185-ae3b-e5c9b336896f\") " pod="openshift-network-operator/iptables-alerter-jn4w2" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186654 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-kubelet-config\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186671 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186701 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-run-multus-certs\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186718 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186733 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0650e893-0096-4562-8d1e-aa06dc59efc0-ovnkube-script-lib\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186748 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-device-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186766 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5506188d-51d5-4bfa-b204-a8e580d3f1b2-serviceca\") pod \"node-ca-5lpws\" (UID: \"5506188d-51d5-4bfa-b204-a8e580d3f1b2\") " pod="openshift-image-registry/node-ca-5lpws" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186796 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-sysctl-d\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186828 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-system-cni-dir\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186843 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-var-lib-cni-bin\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186863 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-var-lib-cni-multus\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186896 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-cnibin\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186947 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-cni-netd\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.186973 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-etc-selinux\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187006 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-multus-cni-dir\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187042 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c69f371c-2aba-473f-8df1-01b116efe170-multus-daemon-config\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.187343 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187069 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-etc-kubernetes\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187094 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/42cde026-7512-4487-a633-846e325b55cc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187121 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-lib-modules\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187220 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/42cde026-7512-4487-a633-846e325b55cc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187248 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-run-netns\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187273 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-run-openvswitch\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187302 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx5nm\" (UniqueName: \"kubernetes.io/projected/0650e893-0096-4562-8d1e-aa06dc59efc0-kube-api-access-cx5nm\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187326 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-dbus\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187352 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-modprobe-d\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187385 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-sysctl-conf\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187431 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9495893a-b14e-4fd9-8999-226d2b5d4eea-hosts-file\") pod \"node-resolver-mkm94\" (UID: \"9495893a-b14e-4fd9-8999-226d2b5d4eea\") " pod="openshift-dns/node-resolver-mkm94" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187490 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-var-lib-openvswitch\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187516 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187552 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpc2m\" (UniqueName: \"kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m\") pod \"network-check-target-7rq9h\" (UID: \"8ba86165-95a0-400b-8168-06860a6ad13a\") " pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187582 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-sys-fs\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.187956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187617 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-multus-socket-dir-parent\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.188708 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187657 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsfx5\" (UniqueName: \"kubernetes.io/projected/4c669f65-9a1b-4a28-9e75-cf0073318a87-kube-api-access-rsfx5\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.188708 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.187879 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-run\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.188708 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.188192 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 31 19:23:27.188880 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.188843 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-sysconfig\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.188967 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.188947 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9495893a-b14e-4fd9-8999-226d2b5d4eea-tmp-dir\") pod \"node-resolver-mkm94\" (UID: \"9495893a-b14e-4fd9-8999-226d2b5d4eea\") " pod="openshift-dns/node-resolver-mkm94" Jul 31 19:23:27.189143 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.189101 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-systemd\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.189750 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.189397 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:27.189750 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.189352 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-host\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.189750 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.189452 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-sysctl-d\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.189750 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.189555 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-kubernetes\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.189750 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.189645 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs podName:b9991931-cf33-4a99-8185-b3ba855761bc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:27.689600469 +0000 UTC m=+2.960947821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs") pod "network-metrics-daemon-9qvpw" (UID: "b9991931-cf33-4a99-8185-b3ba855761bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:27.190075 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.189859 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-var-lib-kubelet\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.190075 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.189952 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-sys\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.190075 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.189989 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-modprobe-d\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.190229 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.190074 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-lib-modules\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.190229 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.190095 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9495893a-b14e-4fd9-8999-226d2b5d4eea-hosts-file\") pod \"node-resolver-mkm94\" (UID: \"9495893a-b14e-4fd9-8999-226d2b5d4eea\") " pod="openshift-dns/node-resolver-mkm94" Jul 31 19:23:27.190229 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.190100 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-sysctl-conf\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.195824 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.193800 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a74545a8-857f-4388-9bd0-4b015d5acaf7-konnectivity-ca\") pod \"konnectivity-agent-hpnqm\" (UID: \"a74545a8-857f-4388-9bd0-4b015d5acaf7\") " pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:27.195824 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.194098 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e36226b-6f84-4535-83f8-f3003a0ee2cd-etc-tuned\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.195824 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.194271 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e36226b-6f84-4535-83f8-f3003a0ee2cd-tmp\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.195824 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.194972 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a74545a8-857f-4388-9bd0-4b015d5acaf7-agent-certs\") pod \"konnectivity-agent-hpnqm\" (UID: \"a74545a8-857f-4388-9bd0-4b015d5acaf7\") " pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:27.202022 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.201962 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpm8z\" (UniqueName: \"kubernetes.io/projected/6e36226b-6f84-4535-83f8-f3003a0ee2cd-kube-api-access-hpm8z\") pod \"tuned-btslq\" (UID: \"6e36226b-6f84-4535-83f8-f3003a0ee2cd\") " pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.202681 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.202648 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8f6m\" (UniqueName: \"kubernetes.io/projected/b9991931-cf33-4a99-8185-b3ba855761bc-kube-api-access-f8f6m\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:27.203715 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.203693 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzgph\" (UniqueName: \"kubernetes.io/projected/9495893a-b14e-4fd9-8999-226d2b5d4eea-kube-api-access-fzgph\") pod \"node-resolver-mkm94\" (UID: \"9495893a-b14e-4fd9-8999-226d2b5d4eea\") " pod="openshift-dns/node-resolver-mkm94" Jul 31 19:23:27.216585 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.216560 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 19:23:27.265898 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.265855 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-28.ec2.internal" event={"ID":"8d9b7ef00ef8d5b576fd1871ffcc6c95","Type":"ContainerStarted","Data":"9c7b7b2f8e3f989ace08079fea0a61054f3f65f02d6e4d39ab03bafb83999487"} Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.288908 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-node-log\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.288951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42cde026-7512-4487-a633-846e325b55cc-cni-binary-copy\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.288979 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-systemd-units\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289008 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c69f371c-2aba-473f-8df1-01b116efe170-cni-binary-copy\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289035 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75srk\" (UniqueName: \"kubernetes.io/projected/5506188d-51d5-4bfa-b204-a8e580d3f1b2-kube-api-access-75srk\") pod \"node-ca-5lpws\" (UID: \"5506188d-51d5-4bfa-b204-a8e580d3f1b2\") " pod="openshift-image-registry/node-ca-5lpws" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289064 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-cnibin\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289074 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-systemd-units\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289086 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-run-netns\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289109 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-slash\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289134 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-run-systemd\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289162 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jggvx\" (UniqueName: \"kubernetes.io/projected/42cde026-7512-4487-a633-846e325b55cc-kube-api-access-jggvx\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289188 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-cni-bin\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289214 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0650e893-0096-4562-8d1e-aa06dc59efc0-ovnkube-config\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289240 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-log-socket\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289262 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289297 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.292697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-registration-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289350 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ab3e83f-09c0-4185-ae3b-e5c9b336896f-iptables-alerter-script\") pod \"iptables-alerter-jn4w2\" (UID: \"4ab3e83f-09c0-4185-ae3b-e5c9b336896f\") " pod="openshift-network-operator/iptables-alerter-jn4w2" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289375 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-kubelet-config\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289445 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-run-multus-certs\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289472 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289527 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0650e893-0096-4562-8d1e-aa06dc59efc0-ovnkube-script-lib\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289565 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-device-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289591 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5506188d-51d5-4bfa-b204-a8e580d3f1b2-serviceca\") pod \"node-ca-5lpws\" (UID: \"5506188d-51d5-4bfa-b204-a8e580d3f1b2\") " pod="openshift-image-registry/node-ca-5lpws" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289616 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-system-cni-dir\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289641 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-var-lib-cni-bin\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-var-lib-cni-multus\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289693 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42cde026-7512-4487-a633-846e325b55cc-cni-binary-copy\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289698 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-cnibin\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289737 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-cni-netd\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289760 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289763 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-etc-selinux\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289789 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-multus-cni-dir\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.293644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289823 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c69f371c-2aba-473f-8df1-01b116efe170-multus-daemon-config\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289848 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-etc-kubernetes\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289878 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/42cde026-7512-4487-a633-846e325b55cc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289913 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289944 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/42cde026-7512-4487-a633-846e325b55cc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289031 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-node-log\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.289979 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-run-netns\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290006 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-run-openvswitch\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290035 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx5nm\" (UniqueName: \"kubernetes.io/projected/0650e893-0096-4562-8d1e-aa06dc59efc0-kube-api-access-cx5nm\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290051 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-cnibin\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290071 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-dbus\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290087 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-run-netns\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290111 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-var-lib-openvswitch\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290129 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-slash\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290137 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290157 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-run-systemd\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290163 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpc2m\" (UniqueName: \"kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m\") pod \"network-check-target-7rq9h\" (UID: \"8ba86165-95a0-400b-8168-06860a6ad13a\") " pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:27.294245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290191 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-sys-fs\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290199 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-cni-bin\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-multus-socket-dir-parent\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290248 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsfx5\" (UniqueName: \"kubernetes.io/projected/4c669f65-9a1b-4a28-9e75-cf0073318a87-kube-api-access-rsfx5\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290276 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-run-k8s-cni-cncf-io\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290300 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0650e893-0096-4562-8d1e-aa06dc59efc0-env-overrides\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290326 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-socket-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290351 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ab3e83f-09c0-4185-ae3b-e5c9b336896f-host-slash\") pod \"iptables-alerter-jn4w2\" (UID: \"4ab3e83f-09c0-4185-ae3b-e5c9b336896f\") " pod="openshift-network-operator/iptables-alerter-jn4w2" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290398 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7sdk\" (UniqueName: \"kubernetes.io/projected/4ab3e83f-09c0-4185-ae3b-e5c9b336896f-kube-api-access-v7sdk\") pod \"iptables-alerter-jn4w2\" (UID: \"4ab3e83f-09c0-4185-ae3b-e5c9b336896f\") " pod="openshift-network-operator/iptables-alerter-jn4w2" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290455 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-os-release\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290490 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-multus-conf-dir\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290528 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-kubelet\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-etc-openvswitch\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290589 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-run-ovn\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290637 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-hostroot\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290657 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0650e893-0096-4562-8d1e-aa06dc59efc0-ovnkube-config\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290664 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0650e893-0096-4562-8d1e-aa06dc59efc0-ovn-node-metrics-cert\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290693 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5506188d-51d5-4bfa-b204-a8e580d3f1b2-host\") pod \"node-ca-5lpws\" (UID: \"5506188d-51d5-4bfa-b204-a8e580d3f1b2\") " pod="openshift-image-registry/node-ca-5lpws" Jul 31 19:23:27.294888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290709 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-log-socket\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290722 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-var-lib-kubelet\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290737 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-run-openvswitch\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290749 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttdgl\" (UniqueName: \"kubernetes.io/projected/c69f371c-2aba-473f-8df1-01b116efe170-kube-api-access-ttdgl\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290774 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290778 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-system-cni-dir\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290820 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-os-release\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.290829 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-system-cni-dir\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291047 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-registration-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291166 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-sys-fs\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291229 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-multus-socket-dir-parent\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291506 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ab3e83f-09c0-4185-ae3b-e5c9b336896f-iptables-alerter-script\") pod \"iptables-alerter-jn4w2\" (UID: \"4ab3e83f-09c0-4185-ae3b-e5c9b336896f\") " pod="openshift-network-operator/iptables-alerter-jn4w2" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291600 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-run-k8s-cni-cncf-io\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291654 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291783 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-run-ovn\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291831 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ab3e83f-09c0-4185-ae3b-e5c9b336896f-host-slash\") pod \"iptables-alerter-jn4w2\" (UID: \"4ab3e83f-09c0-4185-ae3b-e5c9b336896f\") " pod="openshift-network-operator/iptables-alerter-jn4w2" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291865 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-system-cni-dir\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.295559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291888 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-device-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291902 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-os-release\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291917 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-var-lib-cni-bin\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291937 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-multus-conf-dir\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.291978 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-cni-netd\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292008 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0650e893-0096-4562-8d1e-aa06dc59efc0-env-overrides\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292027 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-etc-selinux\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292030 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-kubelet-config\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292048 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-var-lib-kubelet\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292065 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-kubelet\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292070 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-cnibin\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292110 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-host-run-netns\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292140 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-socket-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292154 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-etc-kubernetes\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292203 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42cde026-7512-4487-a633-846e325b55cc-os-release\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292226 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-dbus\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292245 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-hostroot\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292286 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5506188d-51d5-4bfa-b204-a8e580d3f1b2-host\") pod \"node-ca-5lpws\" (UID: \"5506188d-51d5-4bfa-b204-a8e580d3f1b2\") " pod="openshift-image-registry/node-ca-5lpws" Jul 31 19:23:27.296311 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292290 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-run-multus-certs\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292305 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-etc-openvswitch\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292321 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c69f371c-2aba-473f-8df1-01b116efe170-multus-daemon-config\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.292418 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292417 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5506188d-51d5-4bfa-b204-a8e580d3f1b2-serviceca\") pod \"node-ca-5lpws\" (UID: \"5506188d-51d5-4bfa-b204-a8e580d3f1b2\") " pod="openshift-image-registry/node-ca-5lpws" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292456 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-host-var-lib-cni-multus\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.292552 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret podName:0efd55cb-5725-4a4c-9e3b-3725b3041fdc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:27.792533949 +0000 UTC m=+3.063881285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret") pod "global-pull-secret-syncer-xn2mh" (UID: "0efd55cb-5725-4a4c-9e3b-3725b3041fdc") : object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292558 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c669f65-9a1b-4a28-9e75-cf0073318a87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292652 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/42cde026-7512-4487-a633-846e325b55cc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292729 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c69f371c-2aba-473f-8df1-01b116efe170-multus-cni-dir\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292782 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0650e893-0096-4562-8d1e-aa06dc59efc0-var-lib-openvswitch\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292799 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0650e893-0096-4562-8d1e-aa06dc59efc0-ovnkube-script-lib\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.292931 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/42cde026-7512-4487-a633-846e325b55cc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.293274 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c69f371c-2aba-473f-8df1-01b116efe170-cni-binary-copy\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.295342 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0650e893-0096-4562-8d1e-aa06dc59efc0-ovn-node-metrics-cert\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.296524 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.296546 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 19:23:27.297452 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.296561 2565 projected.go:196] Error preparing data for projected volume kube-api-access-tpc2m for pod openshift-network-diagnostics/network-check-target-7rq9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:27.298270 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.296639 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m podName:8ba86165-95a0-400b-8168-06860a6ad13a nodeName:}" failed. No retries permitted until 2026-07-31 19:23:27.796609573 +0000 UTC m=+3.067956922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tpc2m" (UniqueName: "kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m") pod "network-check-target-7rq9h" (UID: "8ba86165-95a0-400b-8168-06860a6ad13a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:27.298819 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.298800 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75srk\" (UniqueName: \"kubernetes.io/projected/5506188d-51d5-4bfa-b204-a8e580d3f1b2-kube-api-access-75srk\") pod \"node-ca-5lpws\" (UID: \"5506188d-51d5-4bfa-b204-a8e580d3f1b2\") " pod="openshift-image-registry/node-ca-5lpws" Jul 31 19:23:27.300051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.300028 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jggvx\" (UniqueName: \"kubernetes.io/projected/42cde026-7512-4487-a633-846e325b55cc-kube-api-access-jggvx\") pod \"multus-additional-cni-plugins-5pmbt\" (UID: \"42cde026-7512-4487-a633-846e325b55cc\") " pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.301417 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.301256 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsfx5\" (UniqueName: \"kubernetes.io/projected/4c669f65-9a1b-4a28-9e75-cf0073318a87-kube-api-access-rsfx5\") pod \"aws-ebs-csi-driver-node-86t69\" (UID: \"4c669f65-9a1b-4a28-9e75-cf0073318a87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.302235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.302205 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttdgl\" (UniqueName: \"kubernetes.io/projected/c69f371c-2aba-473f-8df1-01b116efe170-kube-api-access-ttdgl\") pod \"multus-9762k\" (UID: \"c69f371c-2aba-473f-8df1-01b116efe170\") " pod="openshift-multus/multus-9762k" Jul 31 19:23:27.302731 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.302705 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx5nm\" (UniqueName: \"kubernetes.io/projected/0650e893-0096-4562-8d1e-aa06dc59efc0-kube-api-access-cx5nm\") pod \"ovnkube-node-d2bmz\" (UID: \"0650e893-0096-4562-8d1e-aa06dc59efc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.303109 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.303069 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7sdk\" (UniqueName: \"kubernetes.io/projected/4ab3e83f-09c0-4185-ae3b-e5c9b336896f-kube-api-access-v7sdk\") pod \"iptables-alerter-jn4w2\" (UID: \"4ab3e83f-09c0-4185-ae3b-e5c9b336896f\") " pod="openshift-network-operator/iptables-alerter-jn4w2" Jul 31 19:23:27.384499 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.384460 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-btslq" Jul 31 19:23:27.394473 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.394449 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:27.412391 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.412338 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mkm94" Jul 31 19:23:27.421415 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.421380 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jn4w2" Jul 31 19:23:27.431404 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.431384 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9762k" Jul 31 19:23:27.439247 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.439229 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" Jul 31 19:23:27.449047 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.449021 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5lpws" Jul 31 19:23:27.456905 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.456882 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5pmbt" Jul 31 19:23:27.463878 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.463861 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:27.693336 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.693249 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:27.693508 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.693430 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:27.693508 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.693492 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs podName:b9991931-cf33-4a99-8185-b3ba855761bc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:28.693476621 +0000 UTC m=+3.964823956 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs") pod "network-metrics-daemon-9qvpw" (UID: "b9991931-cf33-4a99-8185-b3ba855761bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:27.794605 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.794563 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:27.794808 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.794727 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:27.794864 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.794809 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret podName:0efd55cb-5725-4a4c-9e3b-3725b3041fdc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:28.794786536 +0000 UTC m=+4.066133874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret") pod "global-pull-secret-syncer-xn2mh" (UID: "0efd55cb-5725-4a4c-9e3b-3725b3041fdc") : object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:27.895907 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:27.895873 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpc2m\" (UniqueName: \"kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m\") pod \"network-check-target-7rq9h\" (UID: \"8ba86165-95a0-400b-8168-06860a6ad13a\") " pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:27.896117 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.896057 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 19:23:27.896117 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.896077 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 19:23:27.896117 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.896087 2565 projected.go:196] Error preparing data for projected volume kube-api-access-tpc2m for pod openshift-network-diagnostics/network-check-target-7rq9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:27.896274 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:27.896134 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m podName:8ba86165-95a0-400b-8168-06860a6ad13a nodeName:}" failed. No retries permitted until 2026-07-31 19:23:28.896120816 +0000 UTC m=+4.167468150 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpc2m" (UniqueName: "kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m") pod "network-check-target-7rq9h" (UID: "8ba86165-95a0-400b-8168-06860a6ad13a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:28.123468 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:28.123408 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 19:18:27 +0000 UTC" deadline="2028-01-22 07:19:36.622513919 +0000 UTC" Jul 31 19:23:28.123468 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:28.123460 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12947h56m8.499058237s" Jul 31 19:23:28.258608 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:28.258568 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:28.258810 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:28.258703 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:28.360887 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:28.360827 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e36226b_6f84_4535_83f8_f3003a0ee2cd.slice/crio-20127e700d6e3b21417a18fabab0600f3619f4d7b853fed6f157f0743c03879d WatchSource:0}: Error finding container 20127e700d6e3b21417a18fabab0600f3619f4d7b853fed6f157f0743c03879d: Status 404 returned error can't find the container with id 20127e700d6e3b21417a18fabab0600f3619f4d7b853fed6f157f0743c03879d Jul 31 19:23:28.365478 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:28.365450 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42cde026_7512_4487_a633_846e325b55cc.slice/crio-ff3e89c25c63200e6cc85c053bbc653a9e52ccc38bf68e20e1f643a3e3f7e2c0 WatchSource:0}: Error finding container ff3e89c25c63200e6cc85c053bbc653a9e52ccc38bf68e20e1f643a3e3f7e2c0: Status 404 returned error can't find the container with id ff3e89c25c63200e6cc85c053bbc653a9e52ccc38bf68e20e1f643a3e3f7e2c0 Jul 31 19:23:28.365830 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:28.365810 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab3e83f_09c0_4185_ae3b_e5c9b336896f.slice/crio-7751de3993717855ddf264658dcee482b5fcfa087275fd1beb924a3307283cdb WatchSource:0}: Error finding container 7751de3993717855ddf264658dcee482b5fcfa087275fd1beb924a3307283cdb: Status 404 returned error can't find the container with id 7751de3993717855ddf264658dcee482b5fcfa087275fd1beb924a3307283cdb Jul 31 19:23:28.380971 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:28.380942 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda74545a8_857f_4388_9bd0_4b015d5acaf7.slice/crio-fb6f353fa75de24bb43ff3d8aa08edbf98ee89f2141d6206453548dfeb1f630d WatchSource:0}: Error finding container fb6f353fa75de24bb43ff3d8aa08edbf98ee89f2141d6206453548dfeb1f630d: Status 404 returned error can't find the container with id fb6f353fa75de24bb43ff3d8aa08edbf98ee89f2141d6206453548dfeb1f630d Jul 31 19:23:28.381432 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:28.381407 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0650e893_0096_4562_8d1e_aa06dc59efc0.slice/crio-5f6f330e0d10f32a1c28c3a92838adb57d12cbdeed8a647737a4bfc3f0a8951f WatchSource:0}: Error finding container 5f6f330e0d10f32a1c28c3a92838adb57d12cbdeed8a647737a4bfc3f0a8951f: Status 404 returned error can't find the container with id 5f6f330e0d10f32a1c28c3a92838adb57d12cbdeed8a647737a4bfc3f0a8951f Jul 31 19:23:28.402867 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:23:28.402846 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9495893a_b14e_4fd9_8999_226d2b5d4eea.slice/crio-d23c590cb12757caa21a3775a53d3b6fecb99fa40af2266b89b0ee970bcfe658 WatchSource:0}: Error finding container d23c590cb12757caa21a3775a53d3b6fecb99fa40af2266b89b0ee970bcfe658: Status 404 returned error can't find the container with id d23c590cb12757caa21a3775a53d3b6fecb99fa40af2266b89b0ee970bcfe658 Jul 31 19:23:28.698681 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:28.698653 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 19:23:28.701405 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:28.701385 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:28.701518 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:28.701508 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:28.701565 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:28.701557 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs podName:b9991931-cf33-4a99-8185-b3ba855761bc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:30.70154285 +0000 UTC m=+5.972890184 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs") pod "network-metrics-daemon-9qvpw" (UID: "b9991931-cf33-4a99-8185-b3ba855761bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:28.801926 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:28.801886 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:28.802113 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:28.802068 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:28.802168 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:28.802129 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret podName:0efd55cb-5725-4a4c-9e3b-3725b3041fdc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:30.802110624 +0000 UTC m=+6.073457977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret") pod "global-pull-secret-syncer-xn2mh" (UID: "0efd55cb-5725-4a4c-9e3b-3725b3041fdc") : object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:28.902584 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:28.902498 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpc2m\" (UniqueName: \"kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m\") pod \"network-check-target-7rq9h\" (UID: \"8ba86165-95a0-400b-8168-06860a6ad13a\") " pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:28.902753 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:28.902648 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 19:23:28.902753 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:28.902664 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 19:23:28.902753 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:28.902675 2565 projected.go:196] Error preparing data for projected volume kube-api-access-tpc2m for pod openshift-network-diagnostics/network-check-target-7rq9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:28.902753 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:28.902724 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m podName:8ba86165-95a0-400b-8168-06860a6ad13a nodeName:}" failed. No retries permitted until 2026-07-31 19:23:30.902710623 +0000 UTC m=+6.174057960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpc2m" (UniqueName: "kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m") pod "network-check-target-7rq9h" (UID: "8ba86165-95a0-400b-8168-06860a6ad13a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:29.124075 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.124005 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 19:18:27 +0000 UTC" deadline="2028-01-15 06:23:24.064729532 +0000 UTC" Jul 31 19:23:29.124075 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.124072 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12778h59m54.940661319s" Jul 31 19:23:29.261008 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.260254 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:29.261008 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:29.260384 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:29.261008 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.260804 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:29.261008 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:29.260900 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:29.275088 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:29.275042 2565 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 500 Internal Server Error; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:13a793596262f904c253062d1a856d35d787f0514effb969ceff913615754fd8" Jul 31 19:23:29.275262 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:29.275154 2565 kuberuntime_manager.go:1459] "Unhandled Error" err="container konnectivity-agent start failed in pod konnectivity-agent-hpnqm_kube-system(a74545a8-857f-4388-9bd0-4b015d5acaf7): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 500 Internal Server Error; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 31 19:23:29.275262 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:29.275202 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 500 Internal Server Error; artifact err: provided artifact is a container image\"" pod="kube-system/konnectivity-agent-hpnqm" podUID="a74545a8-857f-4388-9bd0-4b015d5acaf7" Jul 31 19:23:29.277093 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.277063 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" event={"ID":"0650e893-0096-4562-8d1e-aa06dc59efc0","Type":"ContainerStarted","Data":"5f6f330e0d10f32a1c28c3a92838adb57d12cbdeed8a647737a4bfc3f0a8951f"} Jul 31 19:23:29.281689 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.281654 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" event={"ID":"4c669f65-9a1b-4a28-9e75-cf0073318a87","Type":"ContainerStarted","Data":"116ab02e2129811f4bbfd7d101460168d914651aa2a3e9aeb717461414a21f03"} Jul 31 19:23:29.290567 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.290535 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5lpws" event={"ID":"5506188d-51d5-4bfa-b204-a8e580d3f1b2","Type":"ContainerStarted","Data":"e26029e718c69ae221169da7d63d05c60cf7b2521a334477517ec8bb1fb5fd5a"} Jul 31 19:23:29.295065 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.295032 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jn4w2" event={"ID":"4ab3e83f-09c0-4185-ae3b-e5c9b336896f","Type":"ContainerStarted","Data":"7751de3993717855ddf264658dcee482b5fcfa087275fd1beb924a3307283cdb"} Jul 31 19:23:29.301077 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.301053 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5pmbt" event={"ID":"42cde026-7512-4487-a633-846e325b55cc","Type":"ContainerStarted","Data":"ff3e89c25c63200e6cc85c053bbc653a9e52ccc38bf68e20e1f643a3e3f7e2c0"} Jul 31 19:23:29.309518 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.309490 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-btslq" event={"ID":"6e36226b-6f84-4535-83f8-f3003a0ee2cd","Type":"ContainerStarted","Data":"20127e700d6e3b21417a18fabab0600f3619f4d7b853fed6f157f0743c03879d"} Jul 31 19:23:29.316775 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.316747 2565 generic.go:356] "Generic (PLEG): container finished" podID="fcbfba3810c2bc625686c61579c9f8d1" containerID="a679eedafe726f43a6dc7ee588d5eaf6dbec5d99acde41ddee1bc93693e528df" exitCode=0 Jul 31 19:23:29.316891 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.316833 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" event={"ID":"fcbfba3810c2bc625686c61579c9f8d1","Type":"ContainerDied","Data":"a679eedafe726f43a6dc7ee588d5eaf6dbec5d99acde41ddee1bc93693e528df"} Jul 31 19:23:29.327426 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.327060 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-28.ec2.internal" event={"ID":"8d9b7ef00ef8d5b576fd1871ffcc6c95","Type":"ContainerStarted","Data":"c994b13de7b7bb6284444c5f3ef3c637ecb8033411623171205c075555db4bea"} Jul 31 19:23:29.335258 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.335202 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mkm94" event={"ID":"9495893a-b14e-4fd9-8999-226d2b5d4eea","Type":"ContainerStarted","Data":"d23c590cb12757caa21a3775a53d3b6fecb99fa40af2266b89b0ee970bcfe658"} Jul 31 19:23:29.344064 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.344032 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hpnqm" event={"ID":"a74545a8-857f-4388-9bd0-4b015d5acaf7","Type":"ContainerStarted","Data":"fb6f353fa75de24bb43ff3d8aa08edbf98ee89f2141d6206453548dfeb1f630d"} Jul 31 19:23:29.346913 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:29.346748 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:13a793596262f904c253062d1a856d35d787f0514effb969ceff913615754fd8\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 500 Internal Server Error; artifact err: provided artifact is a container image\"" pod="kube-system/konnectivity-agent-hpnqm" podUID="a74545a8-857f-4388-9bd0-4b015d5acaf7" Jul 31 19:23:29.352535 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.352469 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-28.ec2.internal" podStartSLOduration=2.352450842 podStartE2EDuration="2.352450842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:23:29.351642169 +0000 UTC m=+4.622989526" watchObservedRunningTime="2026-07-31 19:23:29.352450842 +0000 UTC m=+4.623798201" Jul 31 19:23:29.354790 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:29.354661 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9762k" event={"ID":"c69f371c-2aba-473f-8df1-01b116efe170","Type":"ContainerStarted","Data":"14b3c4a9768628d53a5215600f7a0cd9f558b045ccf1d239e72658634fd4804d"} Jul 31 19:23:30.259226 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:30.258671 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:30.259226 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:30.258834 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:30.382609 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:30.382514 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" event={"ID":"fcbfba3810c2bc625686c61579c9f8d1","Type":"ContainerStarted","Data":"a7f532f57da03b5a4f129a12e40461e2075a3856188fda28b85112a950d6047d"} Jul 31 19:23:30.398041 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:30.397981 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-28.ec2.internal" podStartSLOduration=3.397958705 podStartE2EDuration="3.397958705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:23:30.396173267 +0000 UTC m=+5.667520626" watchObservedRunningTime="2026-07-31 19:23:30.397958705 +0000 UTC m=+5.669306064" Jul 31 19:23:30.716728 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:30.716069 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:30.716728 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:30.716270 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:30.716728 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:30.716341 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs podName:b9991931-cf33-4a99-8185-b3ba855761bc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:34.716318124 +0000 UTC m=+9.987665474 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs") pod "network-metrics-daemon-9qvpw" (UID: "b9991931-cf33-4a99-8185-b3ba855761bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:30.817666 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:30.817028 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:30.817666 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:30.817205 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:30.817666 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:30.817271 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret podName:0efd55cb-5725-4a4c-9e3b-3725b3041fdc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:34.817252496 +0000 UTC m=+10.088599832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret") pod "global-pull-secret-syncer-xn2mh" (UID: "0efd55cb-5725-4a4c-9e3b-3725b3041fdc") : object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:30.918164 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:30.918058 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpc2m\" (UniqueName: \"kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m\") pod \"network-check-target-7rq9h\" (UID: \"8ba86165-95a0-400b-8168-06860a6ad13a\") " pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:30.918343 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:30.918286 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 19:23:30.918343 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:30.918306 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 19:23:30.918343 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:30.918319 2565 projected.go:196] Error preparing data for projected volume kube-api-access-tpc2m for pod openshift-network-diagnostics/network-check-target-7rq9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:30.918541 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:30.918383 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m podName:8ba86165-95a0-400b-8168-06860a6ad13a nodeName:}" failed. No retries permitted until 2026-07-31 19:23:34.918363225 +0000 UTC m=+10.189710584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpc2m" (UniqueName: "kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m") pod "network-check-target-7rq9h" (UID: "8ba86165-95a0-400b-8168-06860a6ad13a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:31.258970 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:31.258474 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:31.258970 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:31.258608 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:31.259474 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:31.259299 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:31.259474 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:31.259429 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:32.258724 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:32.258673 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:32.258926 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:32.258826 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:33.261305 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:33.260677 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:33.261305 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:33.260806 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:33.261305 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:33.261156 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:33.261305 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:33.261250 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:34.259048 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:34.258290 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:34.259048 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:34.258463 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:34.751855 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:34.751814 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:34.752332 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:34.752010 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:34.752332 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:34.752090 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs podName:b9991931-cf33-4a99-8185-b3ba855761bc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:42.752074578 +0000 UTC m=+18.023421913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs") pod "network-metrics-daemon-9qvpw" (UID: "b9991931-cf33-4a99-8185-b3ba855761bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:34.852852 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:34.852812 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:34.853031 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:34.852999 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:34.853114 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:34.853077 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret podName:0efd55cb-5725-4a4c-9e3b-3725b3041fdc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:42.853055621 +0000 UTC m=+18.124402962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret") pod "global-pull-secret-syncer-xn2mh" (UID: "0efd55cb-5725-4a4c-9e3b-3725b3041fdc") : object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:34.953751 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:34.953713 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpc2m\" (UniqueName: \"kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m\") pod \"network-check-target-7rq9h\" (UID: \"8ba86165-95a0-400b-8168-06860a6ad13a\") " pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:34.953958 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:34.953874 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 19:23:34.953958 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:34.953897 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 19:23:34.953958 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:34.953910 2565 projected.go:196] Error preparing data for projected volume kube-api-access-tpc2m for pod openshift-network-diagnostics/network-check-target-7rq9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:34.954128 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:34.953979 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m podName:8ba86165-95a0-400b-8168-06860a6ad13a nodeName:}" failed. No retries permitted until 2026-07-31 19:23:42.953959896 +0000 UTC m=+18.225307253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpc2m" (UniqueName: "kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m") pod "network-check-target-7rq9h" (UID: "8ba86165-95a0-400b-8168-06860a6ad13a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:35.260588 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:35.260546 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:35.260779 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:35.260664 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:35.261058 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:35.261034 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:35.261162 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:35.261140 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:36.259022 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:36.258507 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:36.259022 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:36.258654 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:37.258653 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:37.258615 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:37.258858 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:37.258755 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:37.259293 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:37.259148 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:37.259293 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:37.259247 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:38.258771 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:38.258738 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:38.258938 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:38.258855 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:39.257803 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:39.257776 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:39.258163 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:39.257777 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:39.258163 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:39.257876 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:39.258163 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:39.257945 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:40.258720 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:40.258685 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:40.259119 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:40.258878 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:41.258265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:41.258218 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:41.258516 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:41.258474 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:41.258685 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:41.258656 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:41.258831 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:41.258803 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:42.258146 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:42.258116 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:42.258380 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:42.258216 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:42.815464 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:42.815424 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:42.816023 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:42.815609 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:42.816023 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:42.815692 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs podName:b9991931-cf33-4a99-8185-b3ba855761bc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:58.815671166 +0000 UTC m=+34.087018503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs") pod "network-metrics-daemon-9qvpw" (UID: "b9991931-cf33-4a99-8185-b3ba855761bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:42.916618 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:42.916580 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:42.916791 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:42.916732 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:42.916893 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:42.916797 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret podName:0efd55cb-5725-4a4c-9e3b-3725b3041fdc nodeName:}" failed. No retries permitted until 2026-07-31 19:23:58.916776123 +0000 UTC m=+34.188123460 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret") pod "global-pull-secret-syncer-xn2mh" (UID: "0efd55cb-5725-4a4c-9e3b-3725b3041fdc") : object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:43.016964 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:43.016931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpc2m\" (UniqueName: \"kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m\") pod \"network-check-target-7rq9h\" (UID: \"8ba86165-95a0-400b-8168-06860a6ad13a\") " pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:43.017121 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:43.017057 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 19:23:43.017121 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:43.017072 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 19:23:43.017121 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:43.017081 2565 projected.go:196] Error preparing data for projected volume kube-api-access-tpc2m for pod openshift-network-diagnostics/network-check-target-7rq9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:43.017221 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:43.017139 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m podName:8ba86165-95a0-400b-8168-06860a6ad13a nodeName:}" failed. No retries permitted until 2026-07-31 19:23:59.017121275 +0000 UTC m=+34.288468621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpc2m" (UniqueName: "kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m") pod "network-check-target-7rq9h" (UID: "8ba86165-95a0-400b-8168-06860a6ad13a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:43.258736 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:43.258699 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:43.258933 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:43.258832 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:43.258933 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:43.258896 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:43.259048 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:43.259020 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:44.258136 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:44.258098 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:44.258616 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:44.258225 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:45.259699 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:45.259648 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:45.260092 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:45.259768 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:45.260302 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:45.260282 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:45.260425 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:45.260408 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:46.257803 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:46.257767 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:46.258003 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:46.257899 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:47.260434 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:47.260410 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:47.260737 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:47.260507 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:47.260737 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:47.260628 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:47.260737 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:47.260718 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:48.258948 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.258520 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:48.259105 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:48.259000 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:48.418177 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.418142 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mkm94" event={"ID":"9495893a-b14e-4fd9-8999-226d2b5d4eea","Type":"ContainerStarted","Data":"706071b16002e6402b7d832778334343937bf9e67ef4f77094919a331c82f672"} Jul 31 19:23:48.419780 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.419746 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9762k" event={"ID":"c69f371c-2aba-473f-8df1-01b116efe170","Type":"ContainerStarted","Data":"0dd0ed9a2c95b5b63e260dc723fafed48672347627d5847e1231ca6db61215ca"} Jul 31 19:23:48.422101 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.422076 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" event={"ID":"0650e893-0096-4562-8d1e-aa06dc59efc0","Type":"ContainerStarted","Data":"a511547ae0476f0c277c6a6f60cb14c93a633faa68f0ee76b8e0bbcf35d018ae"} Jul 31 19:23:48.422194 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.422139 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" event={"ID":"0650e893-0096-4562-8d1e-aa06dc59efc0","Type":"ContainerStarted","Data":"2f8d789d57df13c54e482ee9c60bca8bab293b46dd20cbd1ada54f2fdaf2f122"} Jul 31 19:23:48.422194 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.422174 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" event={"ID":"0650e893-0096-4562-8d1e-aa06dc59efc0","Type":"ContainerStarted","Data":"07c4fce13a0070368b00a4a2aefa1eb8a8a078e424edfce13d87bb683d740aa1"} Jul 31 19:23:48.422276 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.422208 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" event={"ID":"0650e893-0096-4562-8d1e-aa06dc59efc0","Type":"ContainerStarted","Data":"6430f468c5e808be75d8f9c61b131760068da86e4db8856a9f93a61433cd15c6"} Jul 31 19:23:48.423253 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.423231 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" event={"ID":"4c669f65-9a1b-4a28-9e75-cf0073318a87","Type":"ContainerStarted","Data":"248a85a698b7240ef26ad5f1ddf0fe37f34ede5e674ff2df7b66ba506c464818"} Jul 31 19:23:48.424483 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.424460 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5lpws" event={"ID":"5506188d-51d5-4bfa-b204-a8e580d3f1b2","Type":"ContainerStarted","Data":"ed3214cf9bc892f19c83fd5cd3a2694d99b6be6b34ef28db28e188219e0d059d"} Jul 31 19:23:48.425744 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.425723 2565 generic.go:356] "Generic (PLEG): container finished" podID="42cde026-7512-4487-a633-846e325b55cc" containerID="ddf32d0bbff78477af125d698c3dd24b5982fc9de4c1c76f12113af637cacbba" exitCode=0 Jul 31 19:23:48.425824 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.425779 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5pmbt" event={"ID":"42cde026-7512-4487-a633-846e325b55cc","Type":"ContainerDied","Data":"ddf32d0bbff78477af125d698c3dd24b5982fc9de4c1c76f12113af637cacbba"} Jul 31 19:23:48.427008 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.426989 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-btslq" event={"ID":"6e36226b-6f84-4535-83f8-f3003a0ee2cd","Type":"ContainerStarted","Data":"6e657280a66117ed8bb00b06574a6bf77ad3e39fb6a2f1c6cff12ccb1c3548d9"} Jul 31 19:23:48.432438 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.432400 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mkm94" podStartSLOduration=3.381393526 podStartE2EDuration="22.432388326s" totalImagesPullingTime="19.0509948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:23:48.432151948 +0000 UTC m=+23.703499317" watchObservedRunningTime="2026-07-31 19:23:48.432388326 +0000 UTC m=+23.703735679" Jul 31 19:23:48.446890 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.446856 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9762k" podStartSLOduration=3.239181097 podStartE2EDuration="22.446843148s" totalImagesPullingTime="19.207662051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:23:48.446587687 +0000 UTC m=+23.717935051" watchObservedRunningTime="2026-07-31 19:23:48.446843148 +0000 UTC m=+23.718190501" Jul 31 19:23:48.462070 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.462031 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5lpws" podStartSLOduration=3.587321454 podStartE2EDuration="22.462016461s" totalImagesPullingTime="18.874695007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:23:48.461827165 +0000 UTC m=+23.733174511" watchObservedRunningTime="2026-07-31 19:23:48.462016461 +0000 UTC m=+23.733363847" Jul 31 19:23:48.498595 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:48.498552 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-btslq" podStartSLOduration=3.406969941 podStartE2EDuration="22.498535242s" totalImagesPullingTime="19.091565301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:23:48.497836689 +0000 UTC m=+23.769184043" watchObservedRunningTime="2026-07-31 19:23:48.498535242 +0000 UTC m=+23.769882586" Jul 31 19:23:49.260834 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:49.260805 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:49.261051 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:49.260905 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:49.261051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:49.260961 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:49.261120 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:49.261049 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:49.431153 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:49.431113 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jn4w2" event={"ID":"4ab3e83f-09c0-4185-ae3b-e5c9b336896f","Type":"ContainerStarted","Data":"ba67b5c098f8d6e39db7710282b0b5ca10f25334e22c3ffc35d6c5131c840e27"} Jul 31 19:23:49.432586 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:49.432559 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hpnqm" event={"ID":"a74545a8-857f-4388-9bd0-4b015d5acaf7","Type":"ContainerStarted","Data":"ce9d399a13b3b63a419b890d3c932846314d72cdbe18ad0e863cb617d8e4bb9a"} Jul 31 19:23:49.438946 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:49.438914 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" event={"ID":"0650e893-0096-4562-8d1e-aa06dc59efc0","Type":"ContainerStarted","Data":"ddb374dd30f38da3fa4049fb9f9e2036def4ee910ae24c5b83e93bcb703685fd"} Jul 31 19:23:49.439078 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:49.439002 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" event={"ID":"0650e893-0096-4562-8d1e-aa06dc59efc0","Type":"ContainerStarted","Data":"5b47bef2c80e93c71bf4069ca1324733a1942f33c2f0bb6b619d98126fddf780"} Jul 31 19:23:49.457406 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:49.457331 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jn4w2" podStartSLOduration=4.5825274270000005 podStartE2EDuration="23.457315293s" totalImagesPullingTime="18.874787866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:23:49.443448183 +0000 UTC m=+24.714795539" watchObservedRunningTime="2026-07-31 19:23:49.457315293 +0000 UTC m=+24.728662646" Jul 31 19:23:49.457962 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:49.457931 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hpnqm" podStartSLOduration=3.200316779 podStartE2EDuration="23.45792359s" totalImagesPullingTime="20.257606811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-31 19:23:49.45723422 +0000 UTC m=+24.728581592" watchObservedRunningTime="2026-07-31 19:23:49.45792359 +0000 UTC m=+24.729270943" Jul 31 19:23:49.477018 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:49.476996 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 31 19:23:50.149312 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:50.149191 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-31T19:23:49.477011433Z","UUID":"9aeaddae-106b-4214-80e3-c828ed827f67","Handler":null,"Name":"","Endpoint":""} Jul 31 19:23:50.151095 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:50.151068 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 31 19:23:50.151219 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:50.151117 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 31 19:23:50.257812 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:50.257779 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:50.258043 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:50.257926 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:50.445028 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:50.444936 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" event={"ID":"4c669f65-9a1b-4a28-9e75-cf0073318a87","Type":"ContainerStarted","Data":"07c49b6196590b5f8f217c360f648e62e862fd199c4376820d5325a388103aac"} Jul 31 19:23:50.577313 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:50.577280 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:50.932235 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:50.932199 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:50.932876 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:50.932860 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:51.258403 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:51.258319 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:51.258578 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:51.258330 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:51.259590 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:51.259030 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:51.259590 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:51.259108 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:51.450031 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:51.449990 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" event={"ID":"0650e893-0096-4562-8d1e-aa06dc59efc0","Type":"ContainerStarted","Data":"3d6ec27833320f857b2ef863fddd251520a0d1051cab22bd797a80977d2b410a"} Jul 31 19:23:51.452200 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:51.452171 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" event={"ID":"4c669f65-9a1b-4a28-9e75-cf0073318a87","Type":"ContainerStarted","Data":"84494b2f165345930666a767ecb0cea843588c60789e69fcdb51908f812b4ffd"} Jul 31 19:23:51.452802 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:51.452779 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hpnqm" Jul 31 19:23:51.480239 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:51.480181 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-86t69" podStartSLOduration=3.359895345 podStartE2EDuration="25.480159028s" totalImagesPullingTime="22.120263683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:23:51.467248259 +0000 UTC m=+26.738595613" watchObservedRunningTime="2026-07-31 19:23:51.480159028 +0000 UTC m=+26.751506383" Jul 31 19:23:52.258776 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:52.258742 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:52.258962 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:52.258924 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:53.258156 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:53.257922 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:53.258778 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:53.257961 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:53.258778 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:53.258294 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:53.258778 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:53.258313 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:53.461180 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:53.461148 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" event={"ID":"0650e893-0096-4562-8d1e-aa06dc59efc0","Type":"ContainerStarted","Data":"9a6759847834b2a6803cd93b6f720801218163bb55387d3fcbb4b5ac213ff3fb"} Jul 31 19:23:53.461459 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:53.461439 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:53.461582 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:53.461520 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:53.463334 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:53.463306 2565 generic.go:356] "Generic (PLEG): container finished" podID="42cde026-7512-4487-a633-846e325b55cc" containerID="a63ac358b8b82446e0d6e24e69aa972a98f280cde17546d41de6074696f27882" exitCode=0 Jul 31 19:23:53.463455 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:53.463378 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5pmbt" event={"ID":"42cde026-7512-4487-a633-846e325b55cc","Type":"ContainerDied","Data":"a63ac358b8b82446e0d6e24e69aa972a98f280cde17546d41de6074696f27882"} Jul 31 19:23:53.477846 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:53.477816 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:53.488880 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:53.488821 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" podStartSLOduration=8.39315147 podStartE2EDuration="27.488799131s" totalImagesPullingTime="19.095647661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:23:53.488234057 +0000 UTC m=+28.759581423" watchObservedRunningTime="2026-07-31 19:23:53.488799131 +0000 UTC m=+28.760146485" Jul 31 19:23:54.258252 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:54.258085 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:54.258701 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:54.258370 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:54.465767 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:54.465735 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:54.482314 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:54.482291 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:23:55.261834 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:55.259234 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:55.261834 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:55.259234 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:55.261834 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:55.259421 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:55.261834 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:55.259503 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:55.331091 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:55.331058 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xn2mh"] Jul 31 19:23:55.334074 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:55.333979 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9qvpw"] Jul 31 19:23:55.334233 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:55.334118 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:55.334292 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:55.334226 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:55.334809 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:55.334787 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7rq9h"] Jul 31 19:23:55.469632 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:55.469557 2565 generic.go:356] "Generic (PLEG): container finished" podID="42cde026-7512-4487-a633-846e325b55cc" containerID="2bfbf2bc503aa662ef7d63677bafe35981d848fbd9b21bf1889ece3548b6e440" exitCode=0 Jul 31 19:23:55.469632 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:55.469633 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:55.469799 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:55.469659 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5pmbt" event={"ID":"42cde026-7512-4487-a633-846e325b55cc","Type":"ContainerDied","Data":"2bfbf2bc503aa662ef7d63677bafe35981d848fbd9b21bf1889ece3548b6e440"} Jul 31 19:23:55.469799 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:55.469780 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:55.469931 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:55.469908 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:55.470870 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:55.470301 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:57.258973 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:57.258789 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:57.259378 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:57.258797 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:57.259378 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:57.259085 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:23:57.259378 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:57.259135 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:57.259378 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:57.258798 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:57.259378 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:57.259217 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:57.478995 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:57.478908 2565 generic.go:356] "Generic (PLEG): container finished" podID="42cde026-7512-4487-a633-846e325b55cc" containerID="dbbcce26034fe31d312df9f0a9cac27e72b3647e239b295e1a18012ecb048003" exitCode=0 Jul 31 19:23:57.478995 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:57.478971 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5pmbt" event={"ID":"42cde026-7512-4487-a633-846e325b55cc","Type":"ContainerDied","Data":"dbbcce26034fe31d312df9f0a9cac27e72b3647e239b295e1a18012ecb048003"} Jul 31 19:23:58.832751 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:58.832713 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:58.833159 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:58.832848 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:58.833159 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:58.832906 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs podName:b9991931-cf33-4a99-8185-b3ba855761bc nodeName:}" failed. No retries permitted until 2026-07-31 19:24:30.832890725 +0000 UTC m=+66.104238057 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs") pod "network-metrics-daemon-9qvpw" (UID: "b9991931-cf33-4a99-8185-b3ba855761bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 19:23:58.934074 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:58.934039 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:58.934225 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:58.934170 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:58.934288 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:58.934238 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret podName:0efd55cb-5725-4a4c-9e3b-3725b3041fdc nodeName:}" failed. No retries permitted until 2026-07-31 19:24:30.934218948 +0000 UTC m=+66.205566294 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret") pod "global-pull-secret-syncer-xn2mh" (UID: "0efd55cb-5725-4a4c-9e3b-3725b3041fdc") : object "kube-system"/"original-pull-secret" not registered Jul 31 19:23:59.035254 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:59.035213 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpc2m\" (UniqueName: \"kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m\") pod \"network-check-target-7rq9h\" (UID: \"8ba86165-95a0-400b-8168-06860a6ad13a\") " pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:59.035443 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:59.035380 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 19:23:59.035443 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:59.035398 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 19:23:59.035443 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:59.035406 2565 projected.go:196] Error preparing data for projected volume kube-api-access-tpc2m for pod openshift-network-diagnostics/network-check-target-7rq9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:59.035569 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:59.035461 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m podName:8ba86165-95a0-400b-8168-06860a6ad13a nodeName:}" failed. No retries permitted until 2026-07-31 19:24:31.035445708 +0000 UTC m=+66.306793042 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpc2m" (UniqueName: "kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m") pod "network-check-target-7rq9h" (UID: "8ba86165-95a0-400b-8168-06860a6ad13a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 19:23:59.258663 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:59.258577 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:23:59.258821 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:59.258577 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:23:59.258821 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:23:59.258709 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:23:59.258821 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:59.258702 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xn2mh" podUID="0efd55cb-5725-4a4c-9e3b-3725b3041fdc" Jul 31 19:23:59.258950 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:59.258813 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:23:59.258950 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:23:59.258899 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7rq9h" podUID="8ba86165-95a0-400b-8168-06860a6ad13a" Jul 31 19:24:00.553179 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.553146 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-28.ec2.internal" event="NodeReady" Jul 31 19:24:00.553560 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.553305 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 31 19:24:00.599126 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.599090 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lk42m"] Jul 31 19:24:00.605074 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.605038 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tdqjh"] Jul 31 19:24:00.605241 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.605179 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.607254 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.607225 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 31 19:24:00.609257 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.607300 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 31 19:24:00.609257 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.607372 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w7hsr\"" Jul 31 19:24:00.609257 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.609010 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tdqjh"] Jul 31 19:24:00.609257 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.609156 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tdqjh" Jul 31 19:24:00.609503 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.609470 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lk42m"] Jul 31 19:24:00.611096 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.611073 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 31 19:24:00.611380 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.611335 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rwv9t\"" Jul 31 19:24:00.611380 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.611376 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 31 19:24:00.611548 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.611532 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 31 19:24:00.750678 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.750640 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.750906 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.750689 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-tmp-dir\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.750906 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.750723 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfmv4\" (UniqueName: \"kubernetes.io/projected/9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d-kube-api-access-gfmv4\") pod \"ingress-canary-tdqjh\" (UID: \"9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d\") " pod="openshift-ingress-canary/ingress-canary-tdqjh" Jul 31 19:24:00.750906 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.750776 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d-cert\") pod \"ingress-canary-tdqjh\" (UID: \"9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d\") " pod="openshift-ingress-canary/ingress-canary-tdqjh" Jul 31 19:24:00.750906 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.750827 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khpwm\" (UniqueName: \"kubernetes.io/projected/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-kube-api-access-khpwm\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.750906 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.750852 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-config-volume\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.851959 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.851919 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khpwm\" (UniqueName: \"kubernetes.io/projected/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-kube-api-access-khpwm\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.852143 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.851967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-config-volume\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.852143 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.852015 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.852143 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.852038 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-tmp-dir\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.852143 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.852063 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfmv4\" (UniqueName: \"kubernetes.io/projected/9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d-kube-api-access-gfmv4\") pod \"ingress-canary-tdqjh\" (UID: \"9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d\") " pod="openshift-ingress-canary/ingress-canary-tdqjh" Jul 31 19:24:00.852143 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.852112 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d-cert\") pod \"ingress-canary-tdqjh\" (UID: \"9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d\") " pod="openshift-ingress-canary/ingress-canary-tdqjh" Jul 31 19:24:00.852480 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:00.852159 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 19:24:00.852480 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:00.852228 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls podName:4dd7e20b-fce0-4861-a2b8-39e6e751dfb1 nodeName:}" failed. No retries permitted until 2026-07-31 19:24:01.352208381 +0000 UTC m=+36.623555731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls") pod "dns-default-lk42m" (UID: "4dd7e20b-fce0-4861-a2b8-39e6e751dfb1") : secret "dns-default-metrics-tls" not found Jul 31 19:24:00.852632 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.852613 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-tmp-dir\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.852924 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.852898 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-config-volume\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.857075 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.857052 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d-cert\") pod \"ingress-canary-tdqjh\" (UID: \"9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d\") " pod="openshift-ingress-canary/ingress-canary-tdqjh" Jul 31 19:24:00.859886 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.859867 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khpwm\" (UniqueName: \"kubernetes.io/projected/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-kube-api-access-khpwm\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:00.860415 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.860394 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfmv4\" (UniqueName: \"kubernetes.io/projected/9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d-kube-api-access-gfmv4\") pod \"ingress-canary-tdqjh\" (UID: \"9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d\") " pod="openshift-ingress-canary/ingress-canary-tdqjh" Jul 31 19:24:00.926592 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:00.926557 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tdqjh" Jul 31 19:24:01.258781 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:01.258699 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:24:01.258943 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:01.258699 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:24:01.258943 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:01.258843 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:24:01.261234 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:01.261032 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 31 19:24:01.261234 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:01.261077 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 31 19:24:01.261763 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:01.261742 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 31 19:24:01.261962 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:01.261947 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qbqwg\"" Jul 31 19:24:01.262027 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:01.261970 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 31 19:24:01.262193 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:01.262175 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lpck5\"" Jul 31 19:24:01.356763 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:01.356731 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:01.356936 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:01.356864 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 19:24:01.356936 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:01.356929 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls podName:4dd7e20b-fce0-4861-a2b8-39e6e751dfb1 nodeName:}" failed. No retries permitted until 2026-07-31 19:24:02.356915112 +0000 UTC m=+37.628262448 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls") pod "dns-default-lk42m" (UID: "4dd7e20b-fce0-4861-a2b8-39e6e751dfb1") : secret "dns-default-metrics-tls" not found Jul 31 19:24:01.506099 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:01.506071 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tdqjh"] Jul 31 19:24:01.510826 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:24:01.510600 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fcd2bdb_ce6d_4218_93c6_e40901c0dd2d.slice/crio-1e0399b8fb170a4989d17b57b647c71c4574b61e4b270cf4ced49f70a29a71c0 WatchSource:0}: Error finding container 1e0399b8fb170a4989d17b57b647c71c4574b61e4b270cf4ced49f70a29a71c0: Status 404 returned error can't find the container with id 1e0399b8fb170a4989d17b57b647c71c4574b61e4b270cf4ced49f70a29a71c0 Jul 31 19:24:02.364354 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:02.364157 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:02.365159 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:02.364308 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 19:24:02.365159 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:02.364491 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls podName:4dd7e20b-fce0-4861-a2b8-39e6e751dfb1 nodeName:}" failed. No retries permitted until 2026-07-31 19:24:04.364472868 +0000 UTC m=+39.635820203 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls") pod "dns-default-lk42m" (UID: "4dd7e20b-fce0-4861-a2b8-39e6e751dfb1") : secret "dns-default-metrics-tls" not found Jul 31 19:24:02.493151 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:02.493112 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tdqjh" event={"ID":"9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d","Type":"ContainerStarted","Data":"1e0399b8fb170a4989d17b57b647c71c4574b61e4b270cf4ced49f70a29a71c0"} Jul 31 19:24:02.496082 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:02.496054 2565 generic.go:356] "Generic (PLEG): container finished" podID="42cde026-7512-4487-a633-846e325b55cc" containerID="0c493334076541ac9a3a7c8314d24102fbade265759f71c856c9bfb56fd5c6ee" exitCode=0 Jul 31 19:24:02.496217 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:02.496096 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5pmbt" event={"ID":"42cde026-7512-4487-a633-846e325b55cc","Type":"ContainerDied","Data":"0c493334076541ac9a3a7c8314d24102fbade265759f71c856c9bfb56fd5c6ee"} Jul 31 19:24:03.502755 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:03.502715 2565 generic.go:356] "Generic (PLEG): container finished" podID="42cde026-7512-4487-a633-846e325b55cc" containerID="a56e5baafcc53fe0241b60bc635711bf2492993b1119b87b3c9f1e62b65d748d" exitCode=0 Jul 31 19:24:03.503170 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:03.502771 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5pmbt" event={"ID":"42cde026-7512-4487-a633-846e325b55cc","Type":"ContainerDied","Data":"a56e5baafcc53fe0241b60bc635711bf2492993b1119b87b3c9f1e62b65d748d"} Jul 31 19:24:04.380392 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:04.380333 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:04.380556 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:04.380480 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 19:24:04.380556 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:04.380544 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls podName:4dd7e20b-fce0-4861-a2b8-39e6e751dfb1 nodeName:}" failed. No retries permitted until 2026-07-31 19:24:08.380529329 +0000 UTC m=+43.651876662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls") pod "dns-default-lk42m" (UID: "4dd7e20b-fce0-4861-a2b8-39e6e751dfb1") : secret "dns-default-metrics-tls" not found Jul 31 19:24:04.506578 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:04.506540 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tdqjh" event={"ID":"9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d","Type":"ContainerStarted","Data":"51f6fcad7ab071356418c8a38effffa3a3ad2f6b8a50531aa2e8d3a2374b97d9"} Jul 31 19:24:04.509868 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:04.509842 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5pmbt" event={"ID":"42cde026-7512-4487-a633-846e325b55cc","Type":"ContainerStarted","Data":"c001d7a3e3b83452b26fce62c8736c777202cffa30e8c170d2fa47440d7d8142"} Jul 31 19:24:04.520611 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:04.520566 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tdqjh" podStartSLOduration=2.140956541 podStartE2EDuration="4.520549454s" totalImagesPullingTime="2.379592913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:24:04.519683192 +0000 UTC m=+39.791030545" watchObservedRunningTime="2026-07-31 19:24:04.520549454 +0000 UTC m=+39.791896818" Jul 31 19:24:04.538363 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:04.538300 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5pmbt" podStartSLOduration=8.72944851 podStartE2EDuration="38.53828419s" totalImagesPullingTime="29.80883568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:24:04.537385699 +0000 UTC m=+39.808733053" watchObservedRunningTime="2026-07-31 19:24:04.53828419 +0000 UTC m=+39.809631544" Jul 31 19:24:08.409835 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:08.409660 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:08.409835 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:08.409819 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 19:24:08.410239 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:08.409890 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls podName:4dd7e20b-fce0-4861-a2b8-39e6e751dfb1 nodeName:}" failed. No retries permitted until 2026-07-31 19:24:16.409872669 +0000 UTC m=+51.681220004 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls") pod "dns-default-lk42m" (UID: "4dd7e20b-fce0-4861-a2b8-39e6e751dfb1") : secret "dns-default-metrics-tls" not found Jul 31 19:24:16.463722 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:16.463684 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:16.464286 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:16.463807 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 19:24:16.464286 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:16.463884 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls podName:4dd7e20b-fce0-4861-a2b8-39e6e751dfb1 nodeName:}" failed. No retries permitted until 2026-07-31 19:24:32.463861752 +0000 UTC m=+67.735209088 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls") pod "dns-default-lk42m" (UID: "4dd7e20b-fce0-4861-a2b8-39e6e751dfb1") : secret "dns-default-metrics-tls" not found Jul 31 19:24:26.489202 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:26.489172 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2bmz" Jul 31 19:24:30.868470 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:30.868429 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:24:30.870553 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:30.870537 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 31 19:24:30.879263 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:30.879241 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 31 19:24:30.879371 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:30.879305 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs podName:b9991931-cf33-4a99-8185-b3ba855761bc nodeName:}" failed. No retries permitted until 2026-07-31 19:25:34.879288677 +0000 UTC m=+130.150636011 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs") pod "network-metrics-daemon-9qvpw" (UID: "b9991931-cf33-4a99-8185-b3ba855761bc") : secret "metrics-daemon-secret" not found Jul 31 19:24:30.969171 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:30.969134 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:24:30.971238 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:30.971218 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 31 19:24:30.983696 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:30.983665 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0efd55cb-5725-4a4c-9e3b-3725b3041fdc-original-pull-secret\") pod \"global-pull-secret-syncer-xn2mh\" (UID: \"0efd55cb-5725-4a4c-9e3b-3725b3041fdc\") " pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:24:31.070239 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:31.070204 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpc2m\" (UniqueName: \"kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m\") pod \"network-check-target-7rq9h\" (UID: \"8ba86165-95a0-400b-8168-06860a6ad13a\") " pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:24:31.072334 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:31.072318 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 31 19:24:31.082509 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:31.082492 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 31 19:24:31.093436 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:31.093415 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpc2m\" (UniqueName: \"kubernetes.io/projected/8ba86165-95a0-400b-8168-06860a6ad13a-kube-api-access-tpc2m\") pod \"network-check-target-7rq9h\" (UID: \"8ba86165-95a0-400b-8168-06860a6ad13a\") " pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:24:31.271163 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:31.271086 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xn2mh" Jul 31 19:24:31.285173 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:31.285147 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lpck5\"" Jul 31 19:24:31.293803 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:31.293778 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:24:31.406636 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:31.406599 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xn2mh"] Jul 31 19:24:31.427888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:31.427860 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7rq9h"] Jul 31 19:24:31.431230 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:24:31.431201 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba86165_95a0_400b_8168_06860a6ad13a.slice/crio-61576dd4a0ba84e53c39db41295ba958fd3167db1043222124c71f3962a31d97 WatchSource:0}: Error finding container 61576dd4a0ba84e53c39db41295ba958fd3167db1043222124c71f3962a31d97: Status 404 returned error can't find the container with id 61576dd4a0ba84e53c39db41295ba958fd3167db1043222124c71f3962a31d97 Jul 31 19:24:31.573432 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:31.573392 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7rq9h" event={"ID":"8ba86165-95a0-400b-8168-06860a6ad13a","Type":"ContainerStarted","Data":"61576dd4a0ba84e53c39db41295ba958fd3167db1043222124c71f3962a31d97"} Jul 31 19:24:31.574451 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:31.574428 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xn2mh" event={"ID":"0efd55cb-5725-4a4c-9e3b-3725b3041fdc","Type":"ContainerStarted","Data":"4f976acaf9e60700a49b20ac3f3db290d8b560b1bd3fe035eb53fb665aa1d9d8"} Jul 31 19:24:32.479326 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:32.479279 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:24:32.479804 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:32.479500 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 19:24:32.479804 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:24:32.479565 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls podName:4dd7e20b-fce0-4861-a2b8-39e6e751dfb1 nodeName:}" failed. No retries permitted until 2026-07-31 19:25:04.479546699 +0000 UTC m=+99.750894044 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls") pod "dns-default-lk42m" (UID: "4dd7e20b-fce0-4861-a2b8-39e6e751dfb1") : secret "dns-default-metrics-tls" not found Jul 31 19:24:36.590084 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:36.590049 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7rq9h" event={"ID":"8ba86165-95a0-400b-8168-06860a6ad13a","Type":"ContainerStarted","Data":"b13c6d445bc708d1fd91869b392b2e5a50bc7206e2c6ad5553e8703aad1331e0"} Jul 31 19:24:36.590555 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:36.590171 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:24:36.591594 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:36.591570 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xn2mh" event={"ID":"0efd55cb-5725-4a4c-9e3b-3725b3041fdc","Type":"ContainerStarted","Data":"b26b92b1a2876ba7371d7f1c9bda514ba29b10adfdffb91e250f040ab41c2c74"} Jul 31 19:24:36.603983 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:36.603931 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7rq9h" podStartSLOduration=66.261426344 podStartE2EDuration="1m10.603917268s" totalImagesPullingTime="4.342490924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:24:36.60351158 +0000 UTC m=+71.874858933" watchObservedRunningTime="2026-07-31 19:24:36.603917268 +0000 UTC m=+71.875264621" Jul 31 19:24:36.616294 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:24:36.616243 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xn2mh" podStartSLOduration=66.248605322 podStartE2EDuration="1m10.616228681s" totalImagesPullingTime="4.367623359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:24:36.615630091 +0000 UTC m=+71.886977447" watchObservedRunningTime="2026-07-31 19:24:36.616228681 +0000 UTC m=+71.887576034" Jul 31 19:25:04.489453 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:25:04.489289 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:25:04.489900 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:25:04.489452 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 19:25:04.489900 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:25:04.489525 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls podName:4dd7e20b-fce0-4861-a2b8-39e6e751dfb1 nodeName:}" failed. No retries permitted until 2026-07-31 19:26:08.48950699 +0000 UTC m=+163.760854323 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls") pod "dns-default-lk42m" (UID: "4dd7e20b-fce0-4861-a2b8-39e6e751dfb1") : secret "dns-default-metrics-tls" not found Jul 31 19:25:07.597079 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:25:07.597046 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7rq9h" Jul 31 19:25:34.885741 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:25:34.885682 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:25:34.886216 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:25:34.885835 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 31 19:25:34.886216 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:25:34.885901 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs podName:b9991931-cf33-4a99-8185-b3ba855761bc nodeName:}" failed. No retries permitted until 2026-07-31 19:27:36.885884851 +0000 UTC m=+252.157232183 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs") pod "network-metrics-daemon-9qvpw" (UID: "b9991931-cf33-4a99-8185-b3ba855761bc") : secret "metrics-daemon-secret" not found Jul 31 19:26:03.620300 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:26:03.620257 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lk42m" podUID="4dd7e20b-fce0-4861-a2b8-39e6e751dfb1" Jul 31 19:26:03.804477 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:26:03.804446 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lk42m" Jul 31 19:26:04.278202 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:26:04.278160 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9qvpw" podUID="b9991931-cf33-4a99-8185-b3ba855761bc" Jul 31 19:26:08.515387 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:26:08.515326 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:26:08.515871 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:26:08.515510 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 19:26:08.515871 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:26:08.515607 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls podName:4dd7e20b-fce0-4861-a2b8-39e6e751dfb1 nodeName:}" failed. No retries permitted until 2026-07-31 19:28:10.515586375 +0000 UTC m=+285.786933728 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls") pod "dns-default-lk42m" (UID: "4dd7e20b-fce0-4861-a2b8-39e6e751dfb1") : secret "dns-default-metrics-tls" not found Jul 31 19:26:18.258056 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:26:18.258013 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:27:17.432221 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.432185 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx"] Jul 31 19:27:17.433801 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.433785 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:17.436970 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.436948 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 31 19:27:17.437123 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.437105 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 31 19:27:17.437628 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.437591 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 31 19:27:17.438554 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.438524 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-c2vps\"" Jul 31 19:27:17.438781 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.438765 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-drtx9"] Jul 31 19:27:17.439642 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.439623 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 31 19:27:17.440616 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.440308 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-drtx9" Jul 31 19:27:17.442322 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.442301 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8t2v9\"" Jul 31 19:27:17.442576 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.442561 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 31 19:27:17.442628 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.442592 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 31 19:27:17.455229 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.455202 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx"] Jul 31 19:27:17.455927 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.455909 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-drtx9"] Jul 31 19:27:17.471126 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.471096 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj9vb\" (UniqueName: \"kubernetes.io/projected/649255b5-2ff9-4521-a5c8-01b6cde60ea9-kube-api-access-kj9vb\") pod \"volume-data-source-validator-5b8776669-drtx9\" (UID: \"649255b5-2ff9-4521-a5c8-01b6cde60ea9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-drtx9" Jul 31 19:27:17.471300 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.471143 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:17.471300 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.471181 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:17.471300 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.471214 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqk7d\" (UniqueName: \"kubernetes.io/projected/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-kube-api-access-gqk7d\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:17.527307 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.527278 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-b5t72"] Jul 31 19:27:17.529052 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.529036 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:17.531029 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.531001 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 31 19:27:17.531136 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.531043 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b9zbn\"" Jul 31 19:27:17.531136 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.531009 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 31 19:27:17.533956 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.533936 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm"] Jul 31 19:27:17.535643 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.535627 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg"] Jul 31 19:27:17.535763 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.535745 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:17.537134 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.537118 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-56fb76c566-qls6z"] Jul 31 19:27:17.537418 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.537396 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" Jul 31 19:27:17.537915 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.537845 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 31 19:27:17.538449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.538427 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 31 19:27:17.538542 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.538461 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 31 19:27:17.538542 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.538519 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sr54d\"" Jul 31 19:27:17.539284 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.539269 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-b5t72"] Jul 31 19:27:17.539372 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.539283 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 31 19:27:17.539372 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.539365 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.539559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.539541 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 31 19:27:17.539788 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.539771 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 31 19:27:17.540034 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.540017 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-th5zb\"" Jul 31 19:27:17.540134 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.540107 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 31 19:27:17.541464 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.541421 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 31 19:27:17.541464 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.541438 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 31 19:27:17.541680 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.541665 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 31 19:27:17.541743 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.541692 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 31 19:27:17.541819 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.541800 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 31 19:27:17.541912 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.541827 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 31 19:27:17.541912 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.541892 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9ndzl\"" Jul 31 19:27:17.548838 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.548818 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm"] Jul 31 19:27:17.553935 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.553913 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg"] Jul 31 19:27:17.567875 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.567847 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-56fb76c566-qls6z"] Jul 31 19:27:17.572258 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572237 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-default-certificate\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.572389 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572269 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-stats-auth\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.572389 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj9vb\" (UniqueName: \"kubernetes.io/projected/649255b5-2ff9-4521-a5c8-01b6cde60ea9-kube-api-access-kj9vb\") pod \"volume-data-source-validator-5b8776669-drtx9\" (UID: \"649255b5-2ff9-4521-a5c8-01b6cde60ea9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-drtx9" Jul 31 19:27:17.572389 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572314 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:17.572389 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572336 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:17.572547 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572432 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e977c3b5-b744-453b-9138-8f37b2facb8b-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:17.572547 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572457 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-6hhxm\" (UID: \"ce780b88-ef37-4429-8ead-3839fbc53ed0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:17.572547 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572482 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:17.572547 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572497 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.572685 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:17.572585 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:17.572685 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:17.572644 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls podName:3207a5bd-e828-4d3b-9687-62e26d3b5f3b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:18.072629686 +0000 UTC m=+233.343977021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-98vlx" (UID: "3207a5bd-e828-4d3b-9687-62e26d3b5f3b") : secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:17.572685 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572666 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqk7d\" (UniqueName: \"kubernetes.io/projected/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-kube-api-access-gqk7d\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:17.572819 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572688 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n2v5\" (UniqueName: \"kubernetes.io/projected/ce3db913-8572-4f5b-951f-b4e3ee74f5da-kube-api-access-7n2v5\") pod \"kube-storage-version-migrator-operator-7878f4b88c-x8pbg\" (UID: \"ce3db913-8572-4f5b-951f-b4e3ee74f5da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" Jul 31 19:27:17.572819 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572719 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvvkb\" (UniqueName: \"kubernetes.io/projected/ce780b88-ef37-4429-8ead-3839fbc53ed0-kube-api-access-jvvkb\") pod \"cluster-samples-operator-559cbf5745-6hhxm\" (UID: \"ce780b88-ef37-4429-8ead-3839fbc53ed0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:17.572819 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572734 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ce3db913-8572-4f5b-951f-b4e3ee74f5da-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-x8pbg\" (UID: \"ce3db913-8572-4f5b-951f-b4e3ee74f5da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" Jul 31 19:27:17.572819 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572755 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.572819 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572797 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d259q\" (UniqueName: \"kubernetes.io/projected/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-kube-api-access-d259q\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.573013 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.572827 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ce3db913-8572-4f5b-951f-b4e3ee74f5da-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-x8pbg\" (UID: \"ce3db913-8572-4f5b-951f-b4e3ee74f5da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" Jul 31 19:27:17.573047 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.573025 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:17.584465 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.584434 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqk7d\" (UniqueName: \"kubernetes.io/projected/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-kube-api-access-gqk7d\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:17.584561 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.584509 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj9vb\" (UniqueName: \"kubernetes.io/projected/649255b5-2ff9-4521-a5c8-01b6cde60ea9-kube-api-access-kj9vb\") pod \"volume-data-source-validator-5b8776669-drtx9\" (UID: \"649255b5-2ff9-4521-a5c8-01b6cde60ea9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-drtx9" Jul 31 19:27:17.673929 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.673887 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d259q\" (UniqueName: \"kubernetes.io/projected/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-kube-api-access-d259q\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.673929 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.673935 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ce3db913-8572-4f5b-951f-b4e3ee74f5da-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-x8pbg\" (UID: \"ce3db913-8572-4f5b-951f-b4e3ee74f5da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" Jul 31 19:27:17.674176 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.673960 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-default-certificate\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.674176 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.673992 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-stats-auth\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.674176 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.674020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:17.674176 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.674045 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e977c3b5-b744-453b-9138-8f37b2facb8b-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:17.674176 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.674065 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-6hhxm\" (UID: \"ce780b88-ef37-4429-8ead-3839fbc53ed0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:17.674176 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.674102 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.674176 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:17.674124 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 31 19:27:17.674521 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:17.674194 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert podName:e977c3b5-b744-453b-9138-8f37b2facb8b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:18.174174148 +0000 UTC m=+233.445521488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b5t72" (UID: "e977c3b5-b744-453b-9138-8f37b2facb8b") : secret "networking-console-plugin-cert" not found Jul 31 19:27:17.674521 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:17.674253 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 31 19:27:17.674521 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.674127 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n2v5\" (UniqueName: \"kubernetes.io/projected/ce3db913-8572-4f5b-951f-b4e3ee74f5da-kube-api-access-7n2v5\") pod \"kube-storage-version-migrator-operator-7878f4b88c-x8pbg\" (UID: \"ce3db913-8572-4f5b-951f-b4e3ee74f5da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" Jul 31 19:27:17.674521 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:17.674326 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls podName:ce780b88-ef37-4429-8ead-3839fbc53ed0 nodeName:}" failed. No retries permitted until 2026-07-31 19:27:18.174294517 +0000 UTC m=+233.445641849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-6hhxm" (UID: "ce780b88-ef37-4429-8ead-3839fbc53ed0") : secret "samples-operator-tls" not found Jul 31 19:27:17.674521 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.674406 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvvkb\" (UniqueName: \"kubernetes.io/projected/ce780b88-ef37-4429-8ead-3839fbc53ed0-kube-api-access-jvvkb\") pod \"cluster-samples-operator-559cbf5745-6hhxm\" (UID: \"ce780b88-ef37-4429-8ead-3839fbc53ed0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:17.674521 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.674436 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ce3db913-8572-4f5b-951f-b4e3ee74f5da-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-x8pbg\" (UID: \"ce3db913-8572-4f5b-951f-b4e3ee74f5da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" Jul 31 19:27:17.674521 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:17.674455 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle podName:ff4fac53-98d8-4c8f-b80c-6aec424edf2b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:18.174441416 +0000 UTC m=+233.445788751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle") pod "router-default-56fb76c566-qls6z" (UID: "ff4fac53-98d8-4c8f-b80c-6aec424edf2b") : configmap references non-existent config key: service-ca.crt Jul 31 19:27:17.674521 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.674489 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.674838 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:17.674592 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 31 19:27:17.674838 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:17.674632 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs podName:ff4fac53-98d8-4c8f-b80c-6aec424edf2b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:18.174617147 +0000 UTC m=+233.445964482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs") pod "router-default-56fb76c566-qls6z" (UID: "ff4fac53-98d8-4c8f-b80c-6aec424edf2b") : secret "router-metrics-certs-default" not found Jul 31 19:27:17.674952 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.674932 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ce3db913-8572-4f5b-951f-b4e3ee74f5da-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-x8pbg\" (UID: \"ce3db913-8572-4f5b-951f-b4e3ee74f5da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" Jul 31 19:27:17.675012 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.674998 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e977c3b5-b744-453b-9138-8f37b2facb8b-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:17.676513 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.676498 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ce3db913-8572-4f5b-951f-b4e3ee74f5da-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-x8pbg\" (UID: \"ce3db913-8572-4f5b-951f-b4e3ee74f5da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" Jul 31 19:27:17.677070 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.677053 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-default-certificate\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.677108 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.677053 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-stats-auth\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.681860 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.681838 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d259q\" (UniqueName: \"kubernetes.io/projected/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-kube-api-access-d259q\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:17.682030 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.681869 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n2v5\" (UniqueName: \"kubernetes.io/projected/ce3db913-8572-4f5b-951f-b4e3ee74f5da-kube-api-access-7n2v5\") pod \"kube-storage-version-migrator-operator-7878f4b88c-x8pbg\" (UID: \"ce3db913-8572-4f5b-951f-b4e3ee74f5da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" Jul 31 19:27:17.682091 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.681969 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvvkb\" (UniqueName: \"kubernetes.io/projected/ce780b88-ef37-4429-8ead-3839fbc53ed0-kube-api-access-jvvkb\") pod \"cluster-samples-operator-559cbf5745-6hhxm\" (UID: \"ce780b88-ef37-4429-8ead-3839fbc53ed0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:17.749166 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.749066 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-drtx9" Jul 31 19:27:17.854881 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.854847 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" Jul 31 19:27:17.868025 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.867999 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-drtx9"] Jul 31 19:27:17.872939 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:27:17.872909 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod649255b5_2ff9_4521_a5c8_01b6cde60ea9.slice/crio-0dda0ed0fa2bcd711e3f37a523823df3292e88a560c9e60f7e52113529df7a48 WatchSource:0}: Error finding container 0dda0ed0fa2bcd711e3f37a523823df3292e88a560c9e60f7e52113529df7a48: Status 404 returned error can't find the container with id 0dda0ed0fa2bcd711e3f37a523823df3292e88a560c9e60f7e52113529df7a48 Jul 31 19:27:17.982095 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.982061 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-drtx9" event={"ID":"649255b5-2ff9-4521-a5c8-01b6cde60ea9","Type":"ContainerStarted","Data":"0dda0ed0fa2bcd711e3f37a523823df3292e88a560c9e60f7e52113529df7a48"} Jul 31 19:27:17.982710 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:17.982689 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg"] Jul 31 19:27:17.985501 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:27:17.985481 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce3db913_8572_4f5b_951f_b4e3ee74f5da.slice/crio-8feada659f8803f32889960b20a05ded2b37fa6e999123ad098cefc5df3e8f46 WatchSource:0}: Error finding container 8feada659f8803f32889960b20a05ded2b37fa6e999123ad098cefc5df3e8f46: Status 404 returned error can't find the container with id 8feada659f8803f32889960b20a05ded2b37fa6e999123ad098cefc5df3e8f46 Jul 31 19:27:18.077080 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:18.077051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:18.077232 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:18.077180 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:18.077232 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:18.077229 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls podName:3207a5bd-e828-4d3b-9687-62e26d3b5f3b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:19.077217826 +0000 UTC m=+234.348565159 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-98vlx" (UID: "3207a5bd-e828-4d3b-9687-62e26d3b5f3b") : secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:18.178154 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:18.178117 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:18.178154 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:18.178158 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-6hhxm\" (UID: \"ce780b88-ef37-4429-8ead-3839fbc53ed0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:18.178422 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:18.178196 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:18.178422 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:18.178243 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:18.178422 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:18.178270 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 31 19:27:18.178422 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:18.178335 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert podName:e977c3b5-b744-453b-9138-8f37b2facb8b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:19.1783168 +0000 UTC m=+234.449664132 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b5t72" (UID: "e977c3b5-b744-453b-9138-8f37b2facb8b") : secret "networking-console-plugin-cert" not found Jul 31 19:27:18.178422 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:18.178371 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 31 19:27:18.178628 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:18.178380 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle podName:ff4fac53-98d8-4c8f-b80c-6aec424edf2b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:19.178370982 +0000 UTC m=+234.449718314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle") pod "router-default-56fb76c566-qls6z" (UID: "ff4fac53-98d8-4c8f-b80c-6aec424edf2b") : configmap references non-existent config key: service-ca.crt Jul 31 19:27:18.178628 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:18.178380 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 31 19:27:18.178628 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:18.178496 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls podName:ce780b88-ef37-4429-8ead-3839fbc53ed0 nodeName:}" failed. No retries permitted until 2026-07-31 19:27:19.178479278 +0000 UTC m=+234.449826628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-6hhxm" (UID: "ce780b88-ef37-4429-8ead-3839fbc53ed0") : secret "samples-operator-tls" not found Jul 31 19:27:18.178628 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:18.178515 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs podName:ff4fac53-98d8-4c8f-b80c-6aec424edf2b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:19.178504603 +0000 UTC m=+234.449851946 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs") pod "router-default-56fb76c566-qls6z" (UID: "ff4fac53-98d8-4c8f-b80c-6aec424edf2b") : secret "router-metrics-certs-default" not found Jul 31 19:27:18.986378 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:18.986288 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" event={"ID":"ce3db913-8572-4f5b-951f-b4e3ee74f5da","Type":"ContainerStarted","Data":"8feada659f8803f32889960b20a05ded2b37fa6e999123ad098cefc5df3e8f46"} Jul 31 19:27:19.085594 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:19.085561 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:19.085765 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:19.085711 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:19.085805 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:19.085773 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls podName:3207a5bd-e828-4d3b-9687-62e26d3b5f3b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:21.085758707 +0000 UTC m=+236.357106039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-98vlx" (UID: "3207a5bd-e828-4d3b-9687-62e26d3b5f3b") : secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:19.186184 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:19.186140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:19.186386 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:19.186200 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-6hhxm\" (UID: \"ce780b88-ef37-4429-8ead-3839fbc53ed0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:19.186386 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:19.186236 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:19.186386 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:19.186286 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:19.186386 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:19.186303 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 31 19:27:19.186386 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:19.186384 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert podName:e977c3b5-b744-453b-9138-8f37b2facb8b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:21.186365368 +0000 UTC m=+236.457712700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b5t72" (UID: "e977c3b5-b744-453b-9138-8f37b2facb8b") : secret "networking-console-plugin-cert" not found Jul 31 19:27:19.186630 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:19.186390 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 31 19:27:19.186630 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:19.186417 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 31 19:27:19.186630 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:19.186424 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle podName:ff4fac53-98d8-4c8f-b80c-6aec424edf2b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:21.186404661 +0000 UTC m=+236.457751999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle") pod "router-default-56fb76c566-qls6z" (UID: "ff4fac53-98d8-4c8f-b80c-6aec424edf2b") : configmap references non-existent config key: service-ca.crt Jul 31 19:27:19.186630 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:19.186453 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls podName:ce780b88-ef37-4429-8ead-3839fbc53ed0 nodeName:}" failed. No retries permitted until 2026-07-31 19:27:21.186437678 +0000 UTC m=+236.457785010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-6hhxm" (UID: "ce780b88-ef37-4429-8ead-3839fbc53ed0") : secret "samples-operator-tls" not found Jul 31 19:27:19.186630 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:19.186480 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs podName:ff4fac53-98d8-4c8f-b80c-6aec424edf2b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:21.186470547 +0000 UTC m=+236.457817879 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs") pod "router-default-56fb76c566-qls6z" (UID: "ff4fac53-98d8-4c8f-b80c-6aec424edf2b") : secret "router-metrics-certs-default" not found Jul 31 19:27:19.990469 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:19.990438 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-drtx9" event={"ID":"649255b5-2ff9-4521-a5c8-01b6cde60ea9","Type":"ContainerStarted","Data":"a83dbd2972783a304c1bf46f960ea96f3d4e27541213df935a34e83507c56e56"} Jul 31 19:27:20.004649 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:20.004593 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-drtx9" podStartSLOduration=1.4956220390000001 podStartE2EDuration="3.004577764s" totalImagesPullingTime="1.508955725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:27:20.004006814 +0000 UTC m=+235.275354168" watchObservedRunningTime="2026-07-31 19:27:20.004577764 +0000 UTC m=+235.275925119" Jul 31 19:27:20.933661 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:20.933622 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5552f"] Jul 31 19:27:20.937288 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:20.937265 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:20.939143 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:20.939124 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 31 19:27:20.939143 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:20.939133 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 31 19:27:20.939652 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:20.939638 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 31 19:27:20.939716 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:20.939700 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 31 19:27:20.939760 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:20.939700 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9mcb8\"" Jul 31 19:27:20.945455 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:20.945432 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5552f"] Jul 31 19:27:20.994716 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:20.994685 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" event={"ID":"ce3db913-8572-4f5b-951f-b4e3ee74f5da","Type":"ContainerStarted","Data":"e410eb4006c221eaa3713361fb9a5f4c080b2556ef61fb9de0e80ffa8ac5d8fa"} Jul 31 19:27:21.000906 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.000878 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-crio-socket\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.001031 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.000941 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.001031 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.000960 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.001031 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.000978 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlzwv\" (UniqueName: \"kubernetes.io/projected/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-api-access-wlzwv\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.001031 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.001002 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-data-volume\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.008553 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.008506 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" podStartSLOduration=2.042813457 podStartE2EDuration="4.008489433s" totalImagesPullingTime="1.965675976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:27:21.007589161 +0000 UTC m=+236.278936511" watchObservedRunningTime="2026-07-31 19:27:21.008489433 +0000 UTC m=+236.279836820" Jul 31 19:27:21.101499 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.101450 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-crio-socket\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.101690 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.101533 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.101690 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.101554 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.101690 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.101582 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlzwv\" (UniqueName: \"kubernetes.io/projected/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-api-access-wlzwv\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.101690 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.101593 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-crio-socket\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.101690 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.101611 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-data-volume\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.101690 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.101619 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 19:27:21.101690 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.101676 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm podName:9be97ade-59bb-4c0c-8c59-3a81ab658e8d nodeName:}" failed. No retries permitted until 2026-07-31 19:27:21.601657785 +0000 UTC m=+236.873005122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5552f" (UID: "9be97ade-59bb-4c0c-8c59-3a81ab658e8d") : configmap "kube-rbac-proxy" not found Jul 31 19:27:21.102007 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.101697 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 19:27:21.102007 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.101728 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:21.102007 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.101749 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls podName:9be97ade-59bb-4c0c-8c59-3a81ab658e8d nodeName:}" failed. No retries permitted until 2026-07-31 19:27:21.601733405 +0000 UTC m=+236.873080737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5552f" (UID: "9be97ade-59bb-4c0c-8c59-3a81ab658e8d") : secret "insights-runtime-extractor-tls" not found Jul 31 19:27:21.102007 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.101791 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:21.102007 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.101828 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls podName:3207a5bd-e828-4d3b-9687-62e26d3b5f3b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:25.101818143 +0000 UTC m=+240.373165486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-98vlx" (UID: "3207a5bd-e828-4d3b-9687-62e26d3b5f3b") : secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:21.102007 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.101992 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-data-volume\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.109202 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.109181 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlzwv\" (UniqueName: \"kubernetes.io/projected/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-api-access-wlzwv\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.202511 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.202414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:21.202511 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.202484 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-6hhxm\" (UID: \"ce780b88-ef37-4429-8ead-3839fbc53ed0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:21.202724 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.202556 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:21.202724 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.202576 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 31 19:27:21.202724 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.202585 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:21.202724 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.202591 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 31 19:27:21.202724 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.202641 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs podName:ff4fac53-98d8-4c8f-b80c-6aec424edf2b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:25.202621969 +0000 UTC m=+240.473969304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs") pod "router-default-56fb76c566-qls6z" (UID: "ff4fac53-98d8-4c8f-b80c-6aec424edf2b") : secret "router-metrics-certs-default" not found Jul 31 19:27:21.202724 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.202659 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls podName:ce780b88-ef37-4429-8ead-3839fbc53ed0 nodeName:}" failed. No retries permitted until 2026-07-31 19:27:25.202650215 +0000 UTC m=+240.473997551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-6hhxm" (UID: "ce780b88-ef37-4429-8ead-3839fbc53ed0") : secret "samples-operator-tls" not found Jul 31 19:27:21.202724 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.202662 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 31 19:27:21.202724 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.202706 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle podName:ff4fac53-98d8-4c8f-b80c-6aec424edf2b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:25.202684764 +0000 UTC m=+240.474032100 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle") pod "router-default-56fb76c566-qls6z" (UID: "ff4fac53-98d8-4c8f-b80c-6aec424edf2b") : configmap references non-existent config key: service-ca.crt Jul 31 19:27:21.202724 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.202726 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert podName:e977c3b5-b744-453b-9138-8f37b2facb8b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:25.202719704 +0000 UTC m=+240.474067042 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b5t72" (UID: "e977c3b5-b744-453b-9138-8f37b2facb8b") : secret "networking-console-plugin-cert" not found Jul 31 19:27:21.604897 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.604844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.604897 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:21.604896 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:21.605115 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.604980 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 19:27:21.605115 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.605063 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm podName:9be97ade-59bb-4c0c-8c59-3a81ab658e8d nodeName:}" failed. No retries permitted until 2026-07-31 19:27:22.605042266 +0000 UTC m=+237.876389600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5552f" (UID: "9be97ade-59bb-4c0c-8c59-3a81ab658e8d") : configmap "kube-rbac-proxy" not found Jul 31 19:27:21.605115 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.605072 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 19:27:21.605115 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:21.605108 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls podName:9be97ade-59bb-4c0c-8c59-3a81ab658e8d nodeName:}" failed. No retries permitted until 2026-07-31 19:27:22.605098436 +0000 UTC m=+237.876445768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5552f" (UID: "9be97ade-59bb-4c0c-8c59-3a81ab658e8d") : secret "insights-runtime-extractor-tls" not found Jul 31 19:27:22.613901 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:22.613859 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:22.613901 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:22.613896 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:22.614318 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:22.613985 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 19:27:22.614318 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:22.614003 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 19:27:22.614318 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:22.614052 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm podName:9be97ade-59bb-4c0c-8c59-3a81ab658e8d nodeName:}" failed. No retries permitted until 2026-07-31 19:27:24.614035705 +0000 UTC m=+239.885383040 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5552f" (UID: "9be97ade-59bb-4c0c-8c59-3a81ab658e8d") : configmap "kube-rbac-proxy" not found Jul 31 19:27:22.614318 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:22.614066 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls podName:9be97ade-59bb-4c0c-8c59-3a81ab658e8d nodeName:}" failed. No retries permitted until 2026-07-31 19:27:24.614060054 +0000 UTC m=+239.885407390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5552f" (UID: "9be97ade-59bb-4c0c-8c59-3a81ab658e8d") : secret "insights-runtime-extractor-tls" not found Jul 31 19:27:22.898194 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:22.898126 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mkm94_9495893a-b14e-4fd9-8999-226d2b5d4eea/dns-node-resolver/0.log" Jul 31 19:27:23.697725 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:23.697702 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5lpws_5506188d-51d5-4bfa-b204-a8e580d3f1b2/node-ca/0.log" Jul 31 19:27:24.630788 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:24.630744 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:24.630788 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:24.630791 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:24.631019 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:24.630886 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 19:27:24.631019 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:24.630933 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 19:27:24.631019 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:24.630964 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm podName:9be97ade-59bb-4c0c-8c59-3a81ab658e8d nodeName:}" failed. No retries permitted until 2026-07-31 19:27:28.630946729 +0000 UTC m=+243.902294082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5552f" (UID: "9be97ade-59bb-4c0c-8c59-3a81ab658e8d") : configmap "kube-rbac-proxy" not found Jul 31 19:27:24.631019 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:24.630991 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls podName:9be97ade-59bb-4c0c-8c59-3a81ab658e8d nodeName:}" failed. No retries permitted until 2026-07-31 19:27:28.630971337 +0000 UTC m=+243.902318672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5552f" (UID: "9be97ade-59bb-4c0c-8c59-3a81ab658e8d") : secret "insights-runtime-extractor-tls" not found Jul 31 19:27:24.898831 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:24.898751 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tdqjh_9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d/serve-healthcheck-canary/0.log" Jul 31 19:27:25.135381 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:25.135310 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:25.135556 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:25.135474 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:25.135556 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:25.135545 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls podName:3207a5bd-e828-4d3b-9687-62e26d3b5f3b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:33.135527073 +0000 UTC m=+248.406874405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-98vlx" (UID: "3207a5bd-e828-4d3b-9687-62e26d3b5f3b") : secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:25.236502 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:25.236411 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:25.236502 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:25.236486 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-6hhxm\" (UID: \"ce780b88-ef37-4429-8ead-3839fbc53ed0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:25.236730 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:25.236557 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:25.236730 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:25.236561 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 31 19:27:25.236730 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:25.236582 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:25.236730 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:25.236621 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs podName:ff4fac53-98d8-4c8f-b80c-6aec424edf2b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:33.236606678 +0000 UTC m=+248.507954010 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs") pod "router-default-56fb76c566-qls6z" (UID: "ff4fac53-98d8-4c8f-b80c-6aec424edf2b") : secret "router-metrics-certs-default" not found Jul 31 19:27:25.236730 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:25.236620 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 31 19:27:25.236730 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:25.236678 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 31 19:27:25.236730 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:25.236685 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls podName:ce780b88-ef37-4429-8ead-3839fbc53ed0 nodeName:}" failed. No retries permitted until 2026-07-31 19:27:33.236672957 +0000 UTC m=+248.508020291 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-6hhxm" (UID: "ce780b88-ef37-4429-8ead-3839fbc53ed0") : secret "samples-operator-tls" not found Jul 31 19:27:25.236730 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:25.236729 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert podName:e977c3b5-b744-453b-9138-8f37b2facb8b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:33.236716719 +0000 UTC m=+248.508064084 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b5t72" (UID: "e977c3b5-b744-453b-9138-8f37b2facb8b") : secret "networking-console-plugin-cert" not found Jul 31 19:27:25.236988 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:25.236750 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle podName:ff4fac53-98d8-4c8f-b80c-6aec424edf2b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:33.236736423 +0000 UTC m=+248.508083755 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle") pod "router-default-56fb76c566-qls6z" (UID: "ff4fac53-98d8-4c8f-b80c-6aec424edf2b") : configmap references non-existent config key: service-ca.crt Jul 31 19:27:28.661226 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:28.661189 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:28.661226 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:28.661228 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:28.661711 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:28.661299 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 19:27:28.661711 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:28.661376 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm podName:9be97ade-59bb-4c0c-8c59-3a81ab658e8d nodeName:}" failed. No retries permitted until 2026-07-31 19:27:36.661360887 +0000 UTC m=+251.932708232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5552f" (UID: "9be97ade-59bb-4c0c-8c59-3a81ab658e8d") : configmap "kube-rbac-proxy" not found Jul 31 19:27:28.661711 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:28.661442 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 19:27:28.661711 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:28.661494 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls podName:9be97ade-59bb-4c0c-8c59-3a81ab658e8d nodeName:}" failed. No retries permitted until 2026-07-31 19:27:36.661482516 +0000 UTC m=+251.932829849 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5552f" (UID: "9be97ade-59bb-4c0c-8c59-3a81ab658e8d") : secret "insights-runtime-extractor-tls" not found Jul 31 19:27:33.196801 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.196756 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:33.197160 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:33.196880 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:33.197160 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:33.196934 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls podName:3207a5bd-e828-4d3b-9687-62e26d3b5f3b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:49.196919829 +0000 UTC m=+264.468267161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-98vlx" (UID: "3207a5bd-e828-4d3b-9687-62e26d3b5f3b") : secret "cluster-monitoring-operator-tls" not found Jul 31 19:27:33.297922 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.297886 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:33.298066 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.297966 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-6hhxm\" (UID: \"ce780b88-ef37-4429-8ead-3839fbc53ed0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:33.298066 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.298010 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:33.298066 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.298027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:33.298182 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:33.298125 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 31 19:27:33.298182 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:33.298174 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert podName:e977c3b5-b744-453b-9138-8f37b2facb8b nodeName:}" failed. No retries permitted until 2026-07-31 19:27:49.298160939 +0000 UTC m=+264.569508272 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-b5t72" (UID: "e977c3b5-b744-453b-9138-8f37b2facb8b") : secret "networking-console-plugin-cert" not found Jul 31 19:27:33.298768 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.298698 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-service-ca-bundle\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:33.300519 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.300498 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff4fac53-98d8-4c8f-b80c-6aec424edf2b-metrics-certs\") pod \"router-default-56fb76c566-qls6z\" (UID: \"ff4fac53-98d8-4c8f-b80c-6aec424edf2b\") " pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:33.300620 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.300597 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce780b88-ef37-4429-8ead-3839fbc53ed0-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-6hhxm\" (UID: \"ce780b88-ef37-4429-8ead-3839fbc53ed0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:33.449803 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.449725 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sr54d\"" Jul 31 19:27:33.458168 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.458130 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" Jul 31 19:27:33.462526 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.462322 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9ndzl\"" Jul 31 19:27:33.470952 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.470926 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:33.592633 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.592605 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm"] Jul 31 19:27:33.611204 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:33.611178 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-56fb76c566-qls6z"] Jul 31 19:27:33.613737 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:27:33.613713 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4fac53_98d8_4c8f_b80c_6aec424edf2b.slice/crio-d76a12397cff7dbf893707bc1b9844274eaadcb17022ea859f5b7e171b470ad0 WatchSource:0}: Error finding container d76a12397cff7dbf893707bc1b9844274eaadcb17022ea859f5b7e171b470ad0: Status 404 returned error can't find the container with id d76a12397cff7dbf893707bc1b9844274eaadcb17022ea859f5b7e171b470ad0 Jul 31 19:27:34.028039 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:34.028007 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" event={"ID":"ce780b88-ef37-4429-8ead-3839fbc53ed0","Type":"ContainerStarted","Data":"6d1165027a0682885f25dc6334d36c7e8ff107d83be1a6fdc4233b58f15ae8a5"} Jul 31 19:27:34.029404 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:34.029379 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-56fb76c566-qls6z" event={"ID":"ff4fac53-98d8-4c8f-b80c-6aec424edf2b","Type":"ContainerStarted","Data":"153410dcb9038cadec4dee3217d82a8b83942f7a3c61d4249bc01b4afdc5075d"} Jul 31 19:27:34.029561 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:34.029417 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-56fb76c566-qls6z" event={"ID":"ff4fac53-98d8-4c8f-b80c-6aec424edf2b","Type":"ContainerStarted","Data":"d76a12397cff7dbf893707bc1b9844274eaadcb17022ea859f5b7e171b470ad0"} Jul 31 19:27:34.045629 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:34.045585 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-56fb76c566-qls6z" podStartSLOduration=17.045570438 podStartE2EDuration="17.045570438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:27:34.044933068 +0000 UTC m=+249.316280424" watchObservedRunningTime="2026-07-31 19:27:34.045570438 +0000 UTC m=+249.316917792" Jul 31 19:27:34.472104 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:34.472070 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:34.475106 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:34.475084 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:35.033108 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:35.033078 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:35.034451 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:35.034429 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-56fb76c566-qls6z" Jul 31 19:27:36.037295 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:36.037194 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" event={"ID":"ce780b88-ef37-4429-8ead-3839fbc53ed0","Type":"ContainerStarted","Data":"afc60e0426cea141f2a0459230ea30d284b0cfbc6ca89bba309099257b610460"} Jul 31 19:27:36.037295 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:36.037272 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" event={"ID":"ce780b88-ef37-4429-8ead-3839fbc53ed0","Type":"ContainerStarted","Data":"f900155d9a5cd05f7666bc858545fdae5915ebebd7879d6281e33938d0dd35bc"} Jul 31 19:27:36.052628 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:36.052581 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-6hhxm" podStartSLOduration=17.101391729 podStartE2EDuration="19.052567249s" totalImagesPullingTime="1.95117552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:27:36.051178359 +0000 UTC m=+251.322525715" watchObservedRunningTime="2026-07-31 19:27:36.052567249 +0000 UTC m=+251.323914602" Jul 31 19:27:36.725194 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:36.725149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:36.725194 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:36.725197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:36.725465 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:36.725312 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 19:27:36.725465 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:36.725384 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls podName:9be97ade-59bb-4c0c-8c59-3a81ab658e8d nodeName:}" failed. No retries permitted until 2026-07-31 19:27:52.72537102 +0000 UTC m=+267.996718352 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5552f" (UID: "9be97ade-59bb-4c0c-8c59-3a81ab658e8d") : secret "insights-runtime-extractor-tls" not found Jul 31 19:27:36.725689 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:36.725670 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:36.926457 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:36.926405 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:27:36.928785 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:36.928767 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9991931-cf33-4a99-8185-b3ba855761bc-metrics-certs\") pod \"network-metrics-daemon-9qvpw\" (UID: \"b9991931-cf33-4a99-8185-b3ba855761bc\") " pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:27:37.161476 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:37.161447 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qbqwg\"" Jul 31 19:27:37.170020 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:37.170001 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9qvpw" Jul 31 19:27:37.296753 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:37.296715 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9qvpw"] Jul 31 19:27:37.299416 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:27:37.299390 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9991931_cf33_4a99_8185_b3ba855761bc.slice/crio-119a8d6a91140438711b5095ad784f12ceced9325778ac6fa5db9f5cb49d7037 WatchSource:0}: Error finding container 119a8d6a91140438711b5095ad784f12ceced9325778ac6fa5db9f5cb49d7037: Status 404 returned error can't find the container with id 119a8d6a91140438711b5095ad784f12ceced9325778ac6fa5db9f5cb49d7037 Jul 31 19:27:38.045411 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:38.045373 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9qvpw" event={"ID":"b9991931-cf33-4a99-8185-b3ba855761bc","Type":"ContainerStarted","Data":"119a8d6a91140438711b5095ad784f12ceced9325778ac6fa5db9f5cb49d7037"} Jul 31 19:27:39.050413 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:39.050376 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9qvpw" event={"ID":"b9991931-cf33-4a99-8185-b3ba855761bc","Type":"ContainerStarted","Data":"b8bf742fb8afe98c080addec2a08bbf3f3246f2eb53b4f9970c748b9b9b698c2"} Jul 31 19:27:39.050777 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:39.050433 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9qvpw" event={"ID":"b9991931-cf33-4a99-8185-b3ba855761bc","Type":"ContainerStarted","Data":"c8abd376ce47407ee6648b462b443bf863c9230349749387581cf3861f61bb2b"} Jul 31 19:27:39.066126 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:39.066081 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9qvpw" podStartSLOduration=251.999115483 podStartE2EDuration="4m13.066065008s" totalImagesPullingTime="1.066949525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:27:39.065129041 +0000 UTC m=+254.336476385" watchObservedRunningTime="2026-07-31 19:27:39.066065008 +0000 UTC m=+254.337412362" Jul 31 19:27:49.218915 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:49.218826 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:49.221488 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:49.221463 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3207a5bd-e828-4d3b-9687-62e26d3b5f3b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-98vlx\" (UID: \"3207a5bd-e828-4d3b-9687-62e26d3b5f3b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:49.245159 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:49.245134 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-c2vps\"" Jul 31 19:27:49.253567 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:49.253544 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" Jul 31 19:27:49.319715 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:49.319589 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:49.323408 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:49.323379 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e977c3b5-b744-453b-9138-8f37b2facb8b-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-b5t72\" (UID: \"e977c3b5-b744-453b-9138-8f37b2facb8b\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:49.340810 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:49.340781 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b9zbn\"" Jul 31 19:27:49.349333 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:49.349311 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" Jul 31 19:27:49.383315 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:49.383281 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx"] Jul 31 19:27:49.386914 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:27:49.386888 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3207a5bd_e828_4d3b_9687_62e26d3b5f3b.slice/crio-5f46d118090928303149121b17111b9e6bb8bc61ca5920c3743b86c799c9efa8 WatchSource:0}: Error finding container 5f46d118090928303149121b17111b9e6bb8bc61ca5920c3743b86c799c9efa8: Status 404 returned error can't find the container with id 5f46d118090928303149121b17111b9e6bb8bc61ca5920c3743b86c799c9efa8 Jul 31 19:27:49.480970 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:49.480896 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-b5t72"] Jul 31 19:27:49.483622 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:27:49.483592 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode977c3b5_b744_453b_9138_8f37b2facb8b.slice/crio-b176774868419c9aea06e1b7b74b2e445dc875ab4285637ddc6997e2901f1878 WatchSource:0}: Error finding container b176774868419c9aea06e1b7b74b2e445dc875ab4285637ddc6997e2901f1878: Status 404 returned error can't find the container with id b176774868419c9aea06e1b7b74b2e445dc875ab4285637ddc6997e2901f1878 Jul 31 19:27:50.088261 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:50.088196 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" event={"ID":"e977c3b5-b744-453b-9138-8f37b2facb8b","Type":"ContainerStarted","Data":"b176774868419c9aea06e1b7b74b2e445dc875ab4285637ddc6997e2901f1878"} Jul 31 19:27:50.089533 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:50.089507 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" event={"ID":"3207a5bd-e828-4d3b-9687-62e26d3b5f3b","Type":"ContainerStarted","Data":"5f46d118090928303149121b17111b9e6bb8bc61ca5920c3743b86c799c9efa8"} Jul 31 19:27:51.095359 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:51.095312 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" event={"ID":"e977c3b5-b744-453b-9138-8f37b2facb8b","Type":"ContainerStarted","Data":"89c2e8b2c217df681061189a8d8d02595caf7f1dbaf03a48469105a66779c295"} Jul 31 19:27:51.109610 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:51.109552 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-b5t72" podStartSLOduration=33.044447361 podStartE2EDuration="34.109534987s" totalImagesPullingTime="1.065087626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:27:51.108162578 +0000 UTC m=+266.379509934" watchObservedRunningTime="2026-07-31 19:27:51.109534987 +0000 UTC m=+266.380882340" Jul 31 19:27:52.099773 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:52.099742 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" event={"ID":"3207a5bd-e828-4d3b-9687-62e26d3b5f3b","Type":"ContainerStarted","Data":"3e11ba018e3e9c63599f3c0b533d126fce8ef6905958574633afafa8ac1fbef8"} Jul 31 19:27:52.116483 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:52.116435 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-98vlx" podStartSLOduration=33.320809064 podStartE2EDuration="35.116417514s" totalImagesPullingTime="1.79560845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:27:52.115506431 +0000 UTC m=+267.386853784" watchObservedRunningTime="2026-07-31 19:27:52.116417514 +0000 UTC m=+267.387764867" Jul 31 19:27:52.748151 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:52.748103 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:52.750622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:52.750603 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9be97ade-59bb-4c0c-8c59-3a81ab658e8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5552f\" (UID: \"9be97ade-59bb-4c0c-8c59-3a81ab658e8d\") " pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:53.049483 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:53.049400 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9mcb8\"" Jul 31 19:27:53.057820 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:53.057792 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5552f" Jul 31 19:27:53.184754 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:53.184720 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5552f"] Jul 31 19:27:53.187920 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:27:53.187884 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9be97ade_59bb_4c0c_8c59_3a81ab658e8d.slice/crio-213401bffcd4ada7e8c787126395c1ba65dd27176037e01b0c75f10c4acd8703 WatchSource:0}: Error finding container 213401bffcd4ada7e8c787126395c1ba65dd27176037e01b0c75f10c4acd8703: Status 404 returned error can't find the container with id 213401bffcd4ada7e8c787126395c1ba65dd27176037e01b0c75f10c4acd8703 Jul 31 19:27:54.112059 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:54.111983 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5552f" event={"ID":"9be97ade-59bb-4c0c-8c59-3a81ab658e8d","Type":"ContainerStarted","Data":"2da179f914faba3e66d0b62675bc0a1d49e7aa7b4f064a538d67b751ab8c6a22"} Jul 31 19:27:54.112193 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:54.112059 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5552f" event={"ID":"9be97ade-59bb-4c0c-8c59-3a81ab658e8d","Type":"ContainerStarted","Data":"3de15dbdbcdc232a3377597fc110111f8074c6c4cbf2a69c436b59a78db352bb"} Jul 31 19:27:54.112193 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:54.112078 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5552f" event={"ID":"9be97ade-59bb-4c0c-8c59-3a81ab658e8d","Type":"ContainerStarted","Data":"213401bffcd4ada7e8c787126395c1ba65dd27176037e01b0c75f10c4acd8703"} Jul 31 19:27:56.123935 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:56.123898 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5552f" event={"ID":"9be97ade-59bb-4c0c-8c59-3a81ab658e8d","Type":"ContainerStarted","Data":"12d5d4fe588ca09d2a6f18ff62ce5ff1b96471fed56949700ecf68b9d855abbf"} Jul 31 19:27:56.140479 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:56.140428 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5552f" podStartSLOduration=34.187646614 podStartE2EDuration="36.140412814s" totalImagesPullingTime="1.9527662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:27:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:27:56.139167154 +0000 UTC m=+271.410514508" watchObservedRunningTime="2026-07-31 19:27:56.140412814 +0000 UTC m=+271.411760168" Jul 31 19:27:59.081083 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.081053 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz"] Jul 31 19:27:59.085437 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.085415 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.087753 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.087730 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 31 19:27:59.088564 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.088546 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 31 19:27:59.088666 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.088564 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9d9h7\"" Jul 31 19:27:59.088666 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.088548 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 31 19:27:59.088666 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.088609 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 31 19:27:59.106109 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.106080 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz"] Jul 31 19:27:59.107307 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.107285 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wrz6v"] Jul 31 19:27:59.110508 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.110488 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.112963 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:59.112938 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"node-exporter-accelerators-collector-config\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" Jul 31 19:27:59.112963 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:59.112934 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" Jul 31 19:27:59.113202 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:59.113178 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-dockercfg-zzfsh\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zzfsh\"" type="*v1.Secret" Jul 31 19:27:59.113253 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:59.113189 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-tls\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" Jul 31 19:27:59.197644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197608 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.197644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197646 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.197868 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197675 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-wtmp\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.197868 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197700 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.197868 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197762 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-root\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.197868 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197784 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.197868 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197805 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.197868 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197823 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94lpq\" (UniqueName: \"kubernetes.io/projected/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-api-access-94lpq\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.197868 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197839 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-sys\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.197868 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197855 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.198153 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197933 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-tls\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.198153 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.197975 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.198153 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.198011 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-metrics-client-ca\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.198153 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.198036 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-textfile\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.198153 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.198054 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w54kl\" (UniqueName: \"kubernetes.io/projected/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-kube-api-access-w54kl\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.298639 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.298605 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.298789 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.298669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-metrics-client-ca\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.298789 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.298775 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-textfile\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.298894 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.298812 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w54kl\" (UniqueName: \"kubernetes.io/projected/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-kube-api-access-w54kl\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.298894 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.298834 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.298894 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.298862 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.298894 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.298887 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-wtmp\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.299081 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.298912 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.299081 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.298951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-root\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.299081 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.298985 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.299081 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:59.299011 2565 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 31 19:27:59.299081 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299070 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-textfile\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.299081 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:27:59.299082 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-tls podName:e2fb96e5-3889-4143-85a5-b9dc5d1d6cee nodeName:}" failed. No retries permitted until 2026-07-31 19:27:59.799060346 +0000 UTC m=+275.070407691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-tls") pod "kube-state-metrics-6bd9664c4-zfmgz" (UID: "e2fb96e5-3889-4143-85a5-b9dc5d1d6cee") : secret "kube-state-metrics-tls" not found Jul 31 19:27:59.299362 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299016 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.299362 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299096 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.299362 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299126 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94lpq\" (UniqueName: \"kubernetes.io/projected/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-api-access-94lpq\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.299362 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299159 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-sys\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.299362 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299164 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-wtmp\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.299362 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299187 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.299362 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299235 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-tls\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.299362 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299237 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-sys\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.299362 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299249 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-metrics-client-ca\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.299362 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299158 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-root\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.299814 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299792 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.299899 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.299884 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.301552 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.301529 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.306953 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.306931 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94lpq\" (UniqueName: \"kubernetes.io/projected/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-api-access-94lpq\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.307172 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.307156 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w54kl\" (UniqueName: \"kubernetes.io/projected/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-kube-api-access-w54kl\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:27:59.804209 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.804173 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.806835 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.806808 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2fb96e5-3889-4143-85a5-b9dc5d1d6cee-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-zfmgz\" (UID: \"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:27:59.994008 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:27:59.993974 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" Jul 31 19:28:00.074845 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.074679 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 31 19:28:00.079873 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.079813 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:28:00.130530 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.130500 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz"] Jul 31 19:28:00.133419 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.133396 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 19:28:00.134778 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:28:00.134752 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2fb96e5_3889_4143_85a5_b9dc5d1d6cee.slice/crio-81f0a844ef118c7dfb2fc24c0d87d52f85ab78b0378a98fdd656fc80b5905ab1 WatchSource:0}: Error finding container 81f0a844ef118c7dfb2fc24c0d87d52f85ab78b0378a98fdd656fc80b5905ab1: Status 404 returned error can't find the container with id 81f0a844ef118c7dfb2fc24c0d87d52f85ab78b0378a98fdd656fc80b5905ab1 Jul 31 19:28:00.138172 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.138153 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.140048 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.140028 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 31 19:28:00.140048 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.140040 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 31 19:28:00.140238 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.140073 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 31 19:28:00.140378 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.140362 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 31 19:28:00.140438 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.140389 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 31 19:28:00.140438 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.140407 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 31 19:28:00.140529 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.140439 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 31 19:28:00.140529 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.140467 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 31 19:28:00.140675 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.140662 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 31 19:28:00.140788 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.140771 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-47jkd\"" Jul 31 19:28:00.152960 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.152937 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 19:28:00.207635 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.207605 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.207818 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.207645 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.207818 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.207671 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.207818 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.207700 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-web-config\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.207818 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.207740 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.207818 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.207801 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-config-out\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.208061 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.207871 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.208061 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.207892 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.208061 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.207917 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.208061 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.207932 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6szwx\" (UniqueName: \"kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-kube-api-access-6szwx\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.208061 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.207965 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.208061 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.208002 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.208061 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.208023 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.299356 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:28:00.299324 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-kube-rbac-proxy-config: failed to sync secret cache: timed out waiting for the condition Jul 31 19:28:00.299493 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:28:00.299430 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-kube-rbac-proxy-config podName:97b206c3-b784-4ea5-a74a-cd75d79ba7ad nodeName:}" failed. No retries permitted until 2026-07-31 19:28:00.799412392 +0000 UTC m=+276.070759724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-kube-rbac-proxy-config" (UniqueName: "kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-kube-rbac-proxy-config") pod "node-exporter-wrz6v" (UID: "97b206c3-b784-4ea5-a74a-cd75d79ba7ad") : failed to sync secret cache: timed out waiting for the condition Jul 31 19:28:00.299493 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:28:00.299453 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: failed to sync secret cache: timed out waiting for the condition Jul 31 19:28:00.299568 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:28:00.299528 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-tls podName:97b206c3-b784-4ea5-a74a-cd75d79ba7ad nodeName:}" failed. No retries permitted until 2026-07-31 19:28:00.799511536 +0000 UTC m=+276.070858869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-tls") pod "node-exporter-wrz6v" (UID: "97b206c3-b784-4ea5-a74a-cd75d79ba7ad") : failed to sync secret cache: timed out waiting for the condition Jul 31 19:28:00.309039 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309012 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309111 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309053 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309111 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309077 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6szwx\" (UniqueName: \"kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-kube-api-access-6szwx\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309111 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309095 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309242 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309128 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309242 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309154 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309242 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309202 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309399 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309399 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309281 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309399 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309306 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-web-config\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309399 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309334 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309399 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309380 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-config-out\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309631 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309422 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.309631 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.309440 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.310590 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.310562 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.310753 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:28:00.310700 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-trusted-ca-bundle podName:5103512d-815c-40d8-805e-af4c7f6be3ea nodeName:}" failed. No retries permitted until 2026-07-31 19:28:00.810679383 +0000 UTC m=+276.082026718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea") : configmap references non-existent config key: ca-bundle.crt Jul 31 19:28:00.313160 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.313093 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-web-config\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.313160 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.313093 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.313323 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.313306 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.313528 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.313499 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.313962 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.313942 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.314853 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.314824 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.314962 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.314944 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.315132 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.315110 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-config-out\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.315231 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.315212 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.319738 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.319716 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6szwx\" (UniqueName: \"kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-kube-api-access-6szwx\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.439893 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.439805 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zzfsh\"" Jul 31 19:28:00.493489 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.493463 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 31 19:28:00.552560 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.552532 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 31 19:28:00.814254 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.814168 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.814254 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.814208 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:28:00.814254 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.814250 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-tls\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:28:00.815218 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.815186 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:00.817094 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.817070 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:28:00.817324 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.817305 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97b206c3-b784-4ea5-a74a-cd75d79ba7ad-node-exporter-tls\") pod \"node-exporter-wrz6v\" (UID: \"97b206c3-b784-4ea5-a74a-cd75d79ba7ad\") " pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:28:00.919155 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:00.919122 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wrz6v" Jul 31 19:28:00.930888 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:28:00.930858 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b206c3_b784_4ea5_a74a_cd75d79ba7ad.slice/crio-373d6ce838936cd580f602f97819452230919c1475c9a4c91d2c6b4e9f0270ee WatchSource:0}: Error finding container 373d6ce838936cd580f602f97819452230919c1475c9a4c91d2c6b4e9f0270ee: Status 404 returned error can't find the container with id 373d6ce838936cd580f602f97819452230919c1475c9a4c91d2c6b4e9f0270ee Jul 31 19:28:01.047966 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.047923 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:28:01.055040 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.055011 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-f6db489c-bptgv"] Jul 31 19:28:01.058930 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.058912 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.061023 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.061000 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 31 19:28:01.061128 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.061065 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 31 19:28:01.061858 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.061380 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-en97ak5m7a3q\"" Jul 31 19:28:01.061858 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.061425 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 31 19:28:01.061858 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.061540 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nxhld\"" Jul 31 19:28:01.061858 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.061690 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 31 19:28:01.062251 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.062230 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 31 19:28:01.070495 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.070405 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f6db489c-bptgv"] Jul 31 19:28:01.142916 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.142835 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" event={"ID":"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee","Type":"ContainerStarted","Data":"81f0a844ef118c7dfb2fc24c0d87d52f85ab78b0378a98fdd656fc80b5905ab1"} Jul 31 19:28:01.144648 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.144614 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrz6v" event={"ID":"97b206c3-b784-4ea5-a74a-cd75d79ba7ad","Type":"ContainerStarted","Data":"373d6ce838936cd580f602f97819452230919c1475c9a4c91d2c6b4e9f0270ee"} Jul 31 19:28:01.203401 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.203362 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 19:28:01.216728 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.216686 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.216883 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.216851 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.216936 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.216895 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-grpc-tls\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.216936 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.216921 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.217037 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.216988 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgjss\" (UniqueName: \"kubernetes.io/projected/e1aca7fa-d134-434e-9104-827c90c78f8b-kube-api-access-tgjss\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.217090 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.217049 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.217090 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.217079 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-tls\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.217182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.217104 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1aca7fa-d134-434e-9104-827c90c78f8b-metrics-client-ca\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.318057 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.317961 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.318057 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.318011 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-grpc-tls\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.318057 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.318040 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.318312 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.318067 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgjss\" (UniqueName: \"kubernetes.io/projected/e1aca7fa-d134-434e-9104-827c90c78f8b-kube-api-access-tgjss\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.318312 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.318094 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.318312 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.318118 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-tls\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.318312 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.318141 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1aca7fa-d134-434e-9104-827c90c78f8b-metrics-client-ca\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.318312 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.318179 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.319787 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.319738 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1aca7fa-d134-434e-9104-827c90c78f8b-metrics-client-ca\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.321433 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.321400 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.322102 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.322057 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.322310 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.322291 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-tls\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.322506 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.322486 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.322586 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.322517 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.322730 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.322694 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1aca7fa-d134-434e-9104-827c90c78f8b-secret-grpc-tls\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.326295 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.326268 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgjss\" (UniqueName: \"kubernetes.io/projected/e1aca7fa-d134-434e-9104-827c90c78f8b-kube-api-access-tgjss\") pod \"thanos-querier-f6db489c-bptgv\" (UID: \"e1aca7fa-d134-434e-9104-827c90c78f8b\") " pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.371453 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.371410 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:01.408703 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:28:01.408667 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5103512d_815c_40d8_805e_af4c7f6be3ea.slice/crio-fc7f4b66f4af036ab58727ba3ce829865246dfb48aff6629a4580bf0897c1fad WatchSource:0}: Error finding container fc7f4b66f4af036ab58727ba3ce829865246dfb48aff6629a4580bf0897c1fad: Status 404 returned error can't find the container with id fc7f4b66f4af036ab58727ba3ce829865246dfb48aff6629a4580bf0897c1fad Jul 31 19:28:01.548587 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:01.548489 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f6db489c-bptgv"] Jul 31 19:28:01.554684 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:28:01.554588 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1aca7fa_d134_434e_9104_827c90c78f8b.slice/crio-950b1bd99d7f79d9b42d5818bd88079ab91fd16dc74ab4e10ce98131de354620 WatchSource:0}: Error finding container 950b1bd99d7f79d9b42d5818bd88079ab91fd16dc74ab4e10ce98131de354620: Status 404 returned error can't find the container with id 950b1bd99d7f79d9b42d5818bd88079ab91fd16dc74ab4e10ce98131de354620 Jul 31 19:28:02.151618 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:02.151582 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" event={"ID":"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee","Type":"ContainerStarted","Data":"83a1afe9f53de9f13d94c9e03e1c6a9a072407a39fb53be138e830618a1f2be8"} Jul 31 19:28:02.152140 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:02.151686 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" event={"ID":"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee","Type":"ContainerStarted","Data":"6c5687447796dba81dc96647ec70a1b8e4795dff6a56f67920d95f51c1d9cdb7"} Jul 31 19:28:02.152140 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:02.151717 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" event={"ID":"e2fb96e5-3889-4143-85a5-b9dc5d1d6cee","Type":"ContainerStarted","Data":"8d4814e6dc10765542a55fac7a3b3f74525b213c05cf4a52a437fa9f8588886f"} Jul 31 19:28:02.153005 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:02.152980 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerStarted","Data":"fc7f4b66f4af036ab58727ba3ce829865246dfb48aff6629a4580bf0897c1fad"} Jul 31 19:28:02.154708 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:02.154669 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrz6v" event={"ID":"97b206c3-b784-4ea5-a74a-cd75d79ba7ad","Type":"ContainerStarted","Data":"31bceb745adf1ee3e7d6114922cb655927597a8bbe899d54a28b82ee1634fe64"} Jul 31 19:28:02.155944 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:02.155925 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" event={"ID":"e1aca7fa-d134-434e-9104-827c90c78f8b","Type":"ContainerStarted","Data":"950b1bd99d7f79d9b42d5818bd88079ab91fd16dc74ab4e10ce98131de354620"} Jul 31 19:28:02.168253 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:02.167873 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-zfmgz" podStartSLOduration=1.846451202 podStartE2EDuration="3.167854327s" totalImagesPullingTime="1.321403125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:28:02.167150428 +0000 UTC m=+277.438497782" watchObservedRunningTime="2026-07-31 19:28:02.167854327 +0000 UTC m=+277.439201682" Jul 31 19:28:03.162368 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:03.162308 2565 generic.go:356] "Generic (PLEG): container finished" podID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerID="ee18b49c8c4b34a9d5fb8bc26313260f8cde58882bba228b53644a6a7c644fdf" exitCode=0 Jul 31 19:28:03.162797 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:03.162458 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerDied","Data":"ee18b49c8c4b34a9d5fb8bc26313260f8cde58882bba228b53644a6a7c644fdf"} Jul 31 19:28:03.164407 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:03.164379 2565 generic.go:356] "Generic (PLEG): container finished" podID="97b206c3-b784-4ea5-a74a-cd75d79ba7ad" containerID="31bceb745adf1ee3e7d6114922cb655927597a8bbe899d54a28b82ee1634fe64" exitCode=0 Jul 31 19:28:03.164500 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:03.164467 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrz6v" event={"ID":"97b206c3-b784-4ea5-a74a-cd75d79ba7ad","Type":"ContainerDied","Data":"31bceb745adf1ee3e7d6114922cb655927597a8bbe899d54a28b82ee1634fe64"} Jul 31 19:28:03.842638 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:03.842608 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj"] Jul 31 19:28:03.845760 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:03.845744 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" Jul 31 19:28:03.847625 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:03.847609 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 31 19:28:03.847713 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:03.847698 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5l724\"" Jul 31 19:28:03.852629 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:03.852426 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj"] Jul 31 19:28:03.942626 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:03.942525 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4d7df320-71f8-4fc2-9938-dc27f5e609f1-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-pz4xj\" (UID: \"4d7df320-71f8-4fc2-9938-dc27f5e609f1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" Jul 31 19:28:03.942789 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:03.942685 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x82vk\" (UniqueName: \"kubernetes.io/projected/4d7df320-71f8-4fc2-9938-dc27f5e609f1-kube-api-access-x82vk\") pod \"monitoring-plugin-6dffbbd67-pz4xj\" (UID: \"4d7df320-71f8-4fc2-9938-dc27f5e609f1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" Jul 31 19:28:04.043960 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.043917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x82vk\" (UniqueName: \"kubernetes.io/projected/4d7df320-71f8-4fc2-9938-dc27f5e609f1-kube-api-access-x82vk\") pod \"monitoring-plugin-6dffbbd67-pz4xj\" (UID: \"4d7df320-71f8-4fc2-9938-dc27f5e609f1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" Jul 31 19:28:04.044156 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.043992 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4d7df320-71f8-4fc2-9938-dc27f5e609f1-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-pz4xj\" (UID: \"4d7df320-71f8-4fc2-9938-dc27f5e609f1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" Jul 31 19:28:04.044156 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:28:04.044134 2565 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 31 19:28:04.044254 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:28:04.044193 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d7df320-71f8-4fc2-9938-dc27f5e609f1-monitoring-plugin-cert podName:4d7df320-71f8-4fc2-9938-dc27f5e609f1 nodeName:}" failed. No retries permitted until 2026-07-31 19:28:04.544177645 +0000 UTC m=+279.815524987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/4d7df320-71f8-4fc2-9938-dc27f5e609f1-monitoring-plugin-cert") pod "monitoring-plugin-6dffbbd67-pz4xj" (UID: "4d7df320-71f8-4fc2-9938-dc27f5e609f1") : secret "monitoring-plugin-cert" not found Jul 31 19:28:04.052364 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.052323 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x82vk\" (UniqueName: \"kubernetes.io/projected/4d7df320-71f8-4fc2-9938-dc27f5e609f1-kube-api-access-x82vk\") pod \"monitoring-plugin-6dffbbd67-pz4xj\" (UID: \"4d7df320-71f8-4fc2-9938-dc27f5e609f1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" Jul 31 19:28:04.171910 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.171877 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrz6v" event={"ID":"97b206c3-b784-4ea5-a74a-cd75d79ba7ad","Type":"ContainerStarted","Data":"3994e5bd56284de0ea148ae63f4f35781d89bdc035d448e133002f9a65eb742b"} Jul 31 19:28:04.172396 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.171948 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrz6v" event={"ID":"97b206c3-b784-4ea5-a74a-cd75d79ba7ad","Type":"ContainerStarted","Data":"d853495f4aaa418312bdaf5e843ea7fc1cf1496395a6e4b2e109bb60b93d1fc6"} Jul 31 19:28:04.179630 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.176704 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" event={"ID":"e1aca7fa-d134-434e-9104-827c90c78f8b","Type":"ContainerStarted","Data":"2bdb53aac34b91105dbb31e199b60caa0219c1296667f6f376f3f38674f027be"} Jul 31 19:28:04.179630 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.176829 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" event={"ID":"e1aca7fa-d134-434e-9104-827c90c78f8b","Type":"ContainerStarted","Data":"fc8abd78d1e65b97ec615c447ac0c67aaa9adb5cb42fd6e3b4d195440cef303c"} Jul 31 19:28:04.179630 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.176882 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" event={"ID":"e1aca7fa-d134-434e-9104-827c90c78f8b","Type":"ContainerStarted","Data":"1b8bb0977ba5c296cc2c03b1864e07840483a402ce42d4eea974f076bf531ec8"} Jul 31 19:28:04.190182 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.190089 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wrz6v" podStartSLOduration=4.004495515 podStartE2EDuration="5.190066727s" totalImagesPullingTime="1.111621472s" totalInitContainerRuntime="73.94974ms" isStatefulPod=true podCreationTimestamp="2026-07-31 19:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:28:04.188790437 +0000 UTC m=+279.460137793" watchObservedRunningTime="2026-07-31 19:28:04.190066727 +0000 UTC m=+279.461414083" Jul 31 19:28:04.548104 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.548060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4d7df320-71f8-4fc2-9938-dc27f5e609f1-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-pz4xj\" (UID: \"4d7df320-71f8-4fc2-9938-dc27f5e609f1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" Jul 31 19:28:04.550664 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.550631 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4d7df320-71f8-4fc2-9938-dc27f5e609f1-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-pz4xj\" (UID: \"4d7df320-71f8-4fc2-9938-dc27f5e609f1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" Jul 31 19:28:04.756001 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:04.755969 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" Jul 31 19:28:05.079305 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:05.079278 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj"] Jul 31 19:28:05.082965 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:28:05.082942 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d7df320_71f8_4fc2_9938_dc27f5e609f1.slice/crio-8e73ccc5dd369eafe6a50b3f601a2617398203faaf20d9a1ca5d3a4f0e3b5cc6 WatchSource:0}: Error finding container 8e73ccc5dd369eafe6a50b3f601a2617398203faaf20d9a1ca5d3a4f0e3b5cc6: Status 404 returned error can't find the container with id 8e73ccc5dd369eafe6a50b3f601a2617398203faaf20d9a1ca5d3a4f0e3b5cc6 Jul 31 19:28:05.182644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:05.182606 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" event={"ID":"4d7df320-71f8-4fc2-9938-dc27f5e609f1","Type":"ContainerStarted","Data":"8e73ccc5dd369eafe6a50b3f601a2617398203faaf20d9a1ca5d3a4f0e3b5cc6"} Jul 31 19:28:05.185878 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:05.185849 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerStarted","Data":"a9b0c2c650eccb5bb87f978ffc334b6ef42d36cef7d7a80046b21badce46f86f"} Jul 31 19:28:05.185995 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:05.185935 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerStarted","Data":"499f5f9f809e8b2c8f31fe002f9252f37e3ab0a8161bf0ed6d6eb28db3ab8ff4"} Jul 31 19:28:05.185995 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:05.185964 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerStarted","Data":"6fc85b826da3cf6b55579816c45b4b5dba2c747886cbd57c7681a11a5cfbb2c1"} Jul 31 19:28:05.190085 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:05.189402 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" event={"ID":"e1aca7fa-d134-434e-9104-827c90c78f8b","Type":"ContainerStarted","Data":"85c97c52a8d750b4b1dd9bb392c005594cb98bed3e4ba7c116b2f10f3e68ccd5"} Jul 31 19:28:05.190085 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:05.189465 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" event={"ID":"e1aca7fa-d134-434e-9104-827c90c78f8b","Type":"ContainerStarted","Data":"f1013af939a5fc1c49f86f7e298d50dd5752f173126bcab74c6705a8a15bebca"} Jul 31 19:28:05.190085 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:05.189496 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" event={"ID":"e1aca7fa-d134-434e-9104-827c90c78f8b","Type":"ContainerStarted","Data":"55a7d2e3675b1f7fabc604d054fc4e374a8548c5c82435367e4a839553896130"} Jul 31 19:28:05.216213 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:05.216169 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" podStartSLOduration=1.035104256 podStartE2EDuration="4.216152941s" totalImagesPullingTime="3.181048685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:28:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:28:05.214388367 +0000 UTC m=+280.485735740" watchObservedRunningTime="2026-07-31 19:28:05.216152941 +0000 UTC m=+280.487500294" Jul 31 19:28:06.201537 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:06.201493 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerStarted","Data":"3a7e39b5329890db3a414413fdab92dac330ca5f13cf5d25cd2055c9fe6650b1"} Jul 31 19:28:06.201979 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:06.201604 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerStarted","Data":"d79b976863c2cf8169595d62f4ef99c59af3e2b4e8dc7bea8fb0853161d02336"} Jul 31 19:28:06.201979 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:06.201640 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerStarted","Data":"511d2a16156a7b0ad2cce16d6e352b513f59186852abeb7ffef35fdf8d232fff"} Jul 31 19:28:06.201979 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:06.201824 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:06.225524 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:06.225466 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.080546451 podStartE2EDuration="6.225444067s" totalImagesPullingTime="3.117936479s" totalInitContainerRuntime="26.961137ms" isStatefulPod=false podCreationTimestamp="2026-07-31 19:28:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:28:06.223870719 +0000 UTC m=+281.495218123" watchObservedRunningTime="2026-07-31 19:28:06.225444067 +0000 UTC m=+281.496791422" Jul 31 19:28:06.805842 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:28:06.805797 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lk42m" podUID="4dd7e20b-fce0-4861-a2b8-39e6e751dfb1" Jul 31 19:28:07.206465 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:07.206429 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" event={"ID":"4d7df320-71f8-4fc2-9938-dc27f5e609f1","Type":"ContainerStarted","Data":"ab89ecb4168887e5d7f1a273b55132b77b73d9ffea2b446f25bc833eea7d6353"} Jul 31 19:28:07.206835 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:07.206475 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lk42m" Jul 31 19:28:07.222570 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:07.222523 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" podStartSLOduration=2.722563983 podStartE2EDuration="4.222504563s" totalImagesPullingTime="1.49994058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:28:07.221983823 +0000 UTC m=+282.493331178" watchObservedRunningTime="2026-07-31 19:28:07.222504563 +0000 UTC m=+282.493851916" Jul 31 19:28:08.210623 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:08.210594 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" Jul 31 19:28:08.215314 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:08.215291 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-pz4xj" Jul 31 19:28:10.602152 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:10.602112 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:28:10.604540 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:10.604511 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dd7e20b-fce0-4861-a2b8-39e6e751dfb1-metrics-tls\") pod \"dns-default-lk42m\" (UID: \"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1\") " pod="openshift-dns/dns-default-lk42m" Jul 31 19:28:10.810140 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:10.810109 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w7hsr\"" Jul 31 19:28:10.818825 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:10.818799 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lk42m" Jul 31 19:28:10.947215 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:10.947187 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lk42m"] Jul 31 19:28:10.949833 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:28:10.949806 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd7e20b_fce0_4861_a2b8_39e6e751dfb1.slice/crio-89267e22f38cce9bd5d56b678c65186688059ee853aeaae4a365db88429ef7a1 WatchSource:0}: Error finding container 89267e22f38cce9bd5d56b678c65186688059ee853aeaae4a365db88429ef7a1: Status 404 returned error can't find the container with id 89267e22f38cce9bd5d56b678c65186688059ee853aeaae4a365db88429ef7a1 Jul 31 19:28:11.222545 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:11.222467 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lk42m" event={"ID":"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1","Type":"ContainerStarted","Data":"89267e22f38cce9bd5d56b678c65186688059ee853aeaae4a365db88429ef7a1"} Jul 31 19:28:12.214469 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:12.214424 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-f6db489c-bptgv" Jul 31 19:28:13.232889 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:13.232856 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lk42m" event={"ID":"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1","Type":"ContainerStarted","Data":"2fcf7287cd945cde7bf2ed17b3cb3550cf0fc3bf69f74a8467a9fdb6429026fd"} Jul 31 19:28:13.233268 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:13.232907 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lk42m" event={"ID":"4dd7e20b-fce0-4861-a2b8-39e6e751dfb1","Type":"ContainerStarted","Data":"0d26b9e3132cd426ae527e49396681676b0bc27dbfeccd8ecd2434f8f57b7a92"} Jul 31 19:28:13.233268 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:13.232966 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lk42m" Jul 31 19:28:13.247790 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:13.247743 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lk42m" podStartSLOduration=251.842322436 podStartE2EDuration="4m13.247726699s" totalImagesPullingTime="1.405404263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:28:13.246880023 +0000 UTC m=+288.518227377" watchObservedRunningTime="2026-07-31 19:28:13.247726699 +0000 UTC m=+288.519074053" Jul 31 19:28:23.238693 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:23.238665 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lk42m" Jul 31 19:28:25.109645 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:25.109617 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 31 19:28:26.286498 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:26.286320 2565 generic.go:356] "Generic (PLEG): container finished" podID="ce3db913-8572-4f5b-951f-b4e3ee74f5da" containerID="e410eb4006c221eaa3713361fb9a5f4c080b2556ef61fb9de0e80ffa8ac5d8fa" exitCode=0 Jul 31 19:28:26.286498 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:26.286373 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" event={"ID":"ce3db913-8572-4f5b-951f-b4e3ee74f5da","Type":"ContainerDied","Data":"e410eb4006c221eaa3713361fb9a5f4c080b2556ef61fb9de0e80ffa8ac5d8fa"} Jul 31 19:28:26.286927 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:26.286694 2565 scope.go:117] "RemoveContainer" containerID="e410eb4006c221eaa3713361fb9a5f4c080b2556ef61fb9de0e80ffa8ac5d8fa" Jul 31 19:28:27.291908 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:27.291872 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-x8pbg" event={"ID":"ce3db913-8572-4f5b-951f-b4e3ee74f5da","Type":"ContainerStarted","Data":"fb1932f2b6559941e9b71907412e0e95ede0a611a3ce1f960ea11ffcfaac42ed"} Jul 31 19:28:42.838024 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:42.837998 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5103512d-815c-40d8-805e-af4c7f6be3ea/init-config-reloader/0.log" Jul 31 19:28:43.037474 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:43.037445 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5103512d-815c-40d8-805e-af4c7f6be3ea/alertmanager/0.log" Jul 31 19:28:43.239029 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:43.238948 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5103512d-815c-40d8-805e-af4c7f6be3ea/config-reloader/0.log" Jul 31 19:28:43.437509 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:43.437480 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5103512d-815c-40d8-805e-af4c7f6be3ea/kube-rbac-proxy-web/0.log" Jul 31 19:28:43.637668 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:43.637636 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5103512d-815c-40d8-805e-af4c7f6be3ea/kube-rbac-proxy/0.log" Jul 31 19:28:43.838175 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:43.838145 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5103512d-815c-40d8-805e-af4c7f6be3ea/kube-rbac-proxy-metric/0.log" Jul 31 19:28:44.037456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:44.037376 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5103512d-815c-40d8-805e-af4c7f6be3ea/prom-label-proxy/0.log" Jul 31 19:28:44.239643 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:44.239617 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-98vlx_3207a5bd-e828-4d3b-9687-62e26d3b5f3b/cluster-monitoring-operator/0.log" Jul 31 19:28:44.440417 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:44.440393 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-zfmgz_e2fb96e5-3889-4143-85a5-b9dc5d1d6cee/kube-state-metrics/0.log" Jul 31 19:28:44.637953 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:44.637925 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-zfmgz_e2fb96e5-3889-4143-85a5-b9dc5d1d6cee/kube-rbac-proxy-main/0.log" Jul 31 19:28:44.837972 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:44.837945 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-zfmgz_e2fb96e5-3889-4143-85a5-b9dc5d1d6cee/kube-rbac-proxy-self/0.log" Jul 31 19:28:45.237013 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:45.236937 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-pz4xj_4d7df320-71f8-4fc2-9938-dc27f5e609f1/monitoring-plugin/0.log" Jul 31 19:28:46.636922 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:46.636895 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrz6v_97b206c3-b784-4ea5-a74a-cd75d79ba7ad/init-textfile/0.log" Jul 31 19:28:46.838190 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:46.838162 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrz6v_97b206c3-b784-4ea5-a74a-cd75d79ba7ad/node-exporter/0.log" Jul 31 19:28:47.038786 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:47.038709 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrz6v_97b206c3-b784-4ea5-a74a-cd75d79ba7ad/kube-rbac-proxy/0.log" Jul 31 19:28:50.438466 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:50.438434 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/thanos-query/0.log" Jul 31 19:28:50.637985 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:50.637957 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/kube-rbac-proxy-web/0.log" Jul 31 19:28:50.837579 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:50.837551 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/kube-rbac-proxy/0.log" Jul 31 19:28:51.037246 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:51.037207 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/prom-label-proxy/0.log" Jul 31 19:28:51.237291 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:51.237209 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/kube-rbac-proxy-rules/0.log" Jul 31 19:28:51.438976 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:51.438920 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/kube-rbac-proxy-metrics/0.log" Jul 31 19:28:51.637372 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:51.637307 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-b5t72_e977c3b5-b744-453b-9138-8f37b2facb8b/networking-console-plugin/0.log" Jul 31 19:28:52.839422 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:52.839383 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56fb76c566-qls6z_ff4fac53-98d8-4c8f-b80c-6aec424edf2b/router/0.log" Jul 31 19:28:53.439408 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:53.439367 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tdqjh_9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d/serve-healthcheck-canary/0.log" Jul 31 19:28:53.639911 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:53.639878 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-559cbf5745-6hhxm_ce780b88-ef37-4429-8ead-3839fbc53ed0/cluster-samples-operator/0.log" Jul 31 19:28:53.838658 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:28:53.838624 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-559cbf5745-6hhxm_ce780b88-ef37-4429-8ead-3839fbc53ed0/cluster-samples-operator-watch/0.log" Jul 31 19:29:19.357465 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.357424 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 19:29:19.358029 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.357832 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="alertmanager" containerID="cri-o://6fc85b826da3cf6b55579816c45b4b5dba2c747886cbd57c7681a11a5cfbb2c1" gracePeriod=120 Jul 31 19:29:19.358029 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.357866 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy-web" containerID="cri-o://a9b0c2c650eccb5bb87f978ffc334b6ef42d36cef7d7a80046b21badce46f86f" gracePeriod=120 Jul 31 19:29:19.358029 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.357918 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="prom-label-proxy" containerID="cri-o://3a7e39b5329890db3a414413fdab92dac330ca5f13cf5d25cd2055c9fe6650b1" gracePeriod=120 Jul 31 19:29:19.358029 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.357911 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="config-reloader" containerID="cri-o://499f5f9f809e8b2c8f31fe002f9252f37e3ab0a8161bf0ed6d6eb28db3ab8ff4" gracePeriod=120 Jul 31 19:29:19.358029 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.357900 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy" containerID="cri-o://511d2a16156a7b0ad2cce16d6e352b513f59186852abeb7ffef35fdf8d232fff" gracePeriod=120 Jul 31 19:29:19.358029 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.357866 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy-metric" containerID="cri-o://d79b976863c2cf8169595d62f4ef99c59af3e2b4e8dc7bea8fb0853161d02336" gracePeriod=120 Jul 31 19:29:19.497770 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.497738 2565 generic.go:356] "Generic (PLEG): container finished" podID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerID="3a7e39b5329890db3a414413fdab92dac330ca5f13cf5d25cd2055c9fe6650b1" exitCode=0 Jul 31 19:29:19.497770 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.497762 2565 generic.go:356] "Generic (PLEG): container finished" podID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerID="511d2a16156a7b0ad2cce16d6e352b513f59186852abeb7ffef35fdf8d232fff" exitCode=0 Jul 31 19:29:19.497770 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.497770 2565 generic.go:356] "Generic (PLEG): container finished" podID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerID="499f5f9f809e8b2c8f31fe002f9252f37e3ab0a8161bf0ed6d6eb28db3ab8ff4" exitCode=0 Jul 31 19:29:19.497770 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.497777 2565 generic.go:356] "Generic (PLEG): container finished" podID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerID="6fc85b826da3cf6b55579816c45b4b5dba2c747886cbd57c7681a11a5cfbb2c1" exitCode=0 Jul 31 19:29:19.498012 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.497803 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerDied","Data":"3a7e39b5329890db3a414413fdab92dac330ca5f13cf5d25cd2055c9fe6650b1"} Jul 31 19:29:19.498012 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.497864 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerDied","Data":"511d2a16156a7b0ad2cce16d6e352b513f59186852abeb7ffef35fdf8d232fff"} Jul 31 19:29:19.498012 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.497891 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerDied","Data":"499f5f9f809e8b2c8f31fe002f9252f37e3ab0a8161bf0ed6d6eb28db3ab8ff4"} Jul 31 19:29:19.498012 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:19.497913 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerDied","Data":"6fc85b826da3cf6b55579816c45b4b5dba2c747886cbd57c7681a11a5cfbb2c1"} Jul 31 19:29:20.505704 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.505666 2565 generic.go:356] "Generic (PLEG): container finished" podID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerID="d79b976863c2cf8169595d62f4ef99c59af3e2b4e8dc7bea8fb0853161d02336" exitCode=0 Jul 31 19:29:20.505704 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.505700 2565 generic.go:356] "Generic (PLEG): container finished" podID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerID="a9b0c2c650eccb5bb87f978ffc334b6ef42d36cef7d7a80046b21badce46f86f" exitCode=0 Jul 31 19:29:20.506081 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.505731 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerDied","Data":"d79b976863c2cf8169595d62f4ef99c59af3e2b4e8dc7bea8fb0853161d02336"} Jul 31 19:29:20.506081 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.505797 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerDied","Data":"a9b0c2c650eccb5bb87f978ffc334b6ef42d36cef7d7a80046b21badce46f86f"} Jul 31 19:29:20.604619 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.604595 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:20.655151 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655116 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-main-tls\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655353 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655165 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-metrics-client-ca\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655353 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655198 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655468 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655355 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655468 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655408 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655468 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655440 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-main-db\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655619 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655475 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-config-volume\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655619 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655498 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-web-config\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655619 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655524 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-config-out\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655619 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655554 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-trusted-ca-bundle\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655619 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655561 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 19:29:20.655619 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655581 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-cluster-tls-config\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655900 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655627 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-tls-assets\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.655900 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655701 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6szwx\" (UniqueName: \"kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-kube-api-access-6szwx\") pod \"5103512d-815c-40d8-805e-af4c7f6be3ea\" (UID: \"5103512d-815c-40d8-805e-af4c7f6be3ea\") " Jul 31 19:29:20.656002 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.655899 2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-metrics-client-ca\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.656359 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.656322 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 19:29:20.657154 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.657079 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 19:29:20.659596 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.659565 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-kube-api-access-6szwx" (OuterVolumeSpecName: "kube-api-access-6szwx") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "kube-api-access-6szwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 19:29:20.660145 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.659997 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-config-out" (OuterVolumeSpecName: "config-out") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 19:29:20.660145 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.660027 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 19:29:20.660145 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.660104 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 19:29:20.660145 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.660118 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 19:29:20.660412 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.660336 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-config-volume" (OuterVolumeSpecName: "config-volume") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 19:29:20.660412 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.660372 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 19:29:20.660749 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.660730 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 19:29:20.664528 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.664480 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 19:29:20.671264 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.671244 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-web-config" (OuterVolumeSpecName: "web-config") pod "5103512d-815c-40d8-805e-af4c7f6be3ea" (UID: "5103512d-815c-40d8-805e-af4c7f6be3ea"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 19:29:20.757011 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.756975 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.757167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.757017 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.757167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.757033 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.757167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.757049 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-main-db\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.757167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.757065 2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-config-volume\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.757167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.757079 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-web-config\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.757167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.757091 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5103512d-815c-40d8-805e-af4c7f6be3ea-config-out\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.757167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.757122 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5103512d-815c-40d8-805e-af4c7f6be3ea-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.757167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.757138 2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-cluster-tls-config\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.757167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.757150 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-tls-assets\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.757167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.757163 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6szwx\" (UniqueName: \"kubernetes.io/projected/5103512d-815c-40d8-805e-af4c7f6be3ea-kube-api-access-6szwx\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:20.757472 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:20.757176 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5103512d-815c-40d8-805e-af4c7f6be3ea-secret-alertmanager-main-tls\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:29:21.512914 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.512880 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5103512d-815c-40d8-805e-af4c7f6be3ea","Type":"ContainerDied","Data":"fc7f4b66f4af036ab58727ba3ce829865246dfb48aff6629a4580bf0897c1fad"} Jul 31 19:29:21.513306 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.512955 2565 scope.go:117] "RemoveContainer" containerID="3a7e39b5329890db3a414413fdab92dac330ca5f13cf5d25cd2055c9fe6650b1" Jul 31 19:29:21.513306 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.512897 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.520828 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.520810 2565 scope.go:117] "RemoveContainer" containerID="d79b976863c2cf8169595d62f4ef99c59af3e2b4e8dc7bea8fb0853161d02336" Jul 31 19:29:21.528109 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.528087 2565 scope.go:117] "RemoveContainer" containerID="511d2a16156a7b0ad2cce16d6e352b513f59186852abeb7ffef35fdf8d232fff" Jul 31 19:29:21.530539 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.530512 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 19:29:21.534654 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.534630 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 19:29:21.537049 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.537029 2565 scope.go:117] "RemoveContainer" containerID="a9b0c2c650eccb5bb87f978ffc334b6ef42d36cef7d7a80046b21badce46f86f" Jul 31 19:29:21.544058 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.544042 2565 scope.go:117] "RemoveContainer" containerID="499f5f9f809e8b2c8f31fe002f9252f37e3ab0a8161bf0ed6d6eb28db3ab8ff4" Jul 31 19:29:21.551110 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.551089 2565 scope.go:117] "RemoveContainer" containerID="6fc85b826da3cf6b55579816c45b4b5dba2c747886cbd57c7681a11a5cfbb2c1" Jul 31 19:29:21.559118 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.559051 2565 scope.go:117] "RemoveContainer" containerID="ee18b49c8c4b34a9d5fb8bc26313260f8cde58882bba228b53644a6a7c644fdf" Jul 31 19:29:21.561813 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.561783 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 19:29:21.562068 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562056 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="alertmanager" Jul 31 19:29:21.562113 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562069 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="alertmanager" Jul 31 19:29:21.562113 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562084 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="config-reloader" Jul 31 19:29:21.562113 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562090 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="config-reloader" Jul 31 19:29:21.562113 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562101 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="init-config-reloader" Jul 31 19:29:21.562113 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562107 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="init-config-reloader" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562119 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy-web" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562124 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy-web" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562130 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562135 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562140 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy-metric" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562145 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy-metric" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562152 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="prom-label-proxy" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562157 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="prom-label-proxy" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562197 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy-metric" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562204 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="prom-label-proxy" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562209 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562216 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="config-reloader" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562222 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="alertmanager" Jul 31 19:29:21.562265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.562228 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" containerName="kube-rbac-proxy-web" Jul 31 19:29:21.567428 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.567411 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.569469 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.569444 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 31 19:29:21.569614 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.569489 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 31 19:29:21.569614 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.569496 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 31 19:29:21.569614 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.569543 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 31 19:29:21.569614 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.569560 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 31 19:29:21.569828 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.569739 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 31 19:29:21.569828 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.569755 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 31 19:29:21.569828 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.569772 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 31 19:29:21.569828 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.569810 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-47jkd\"" Jul 31 19:29:21.577387 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.575649 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 31 19:29:21.580277 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.580251 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 19:29:21.662566 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662529 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-web-config\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.662566 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662568 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.662782 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662587 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.662782 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662611 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.662782 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662658 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57f53419-1aeb-4cae-8912-604df35afc71-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.662782 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662698 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57f53419-1aeb-4cae-8912-604df35afc71-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.662782 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57f53419-1aeb-4cae-8912-604df35afc71-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.662782 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662736 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg9r4\" (UniqueName: \"kubernetes.io/projected/57f53419-1aeb-4cae-8912-604df35afc71-kube-api-access-gg9r4\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.662782 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662761 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57f53419-1aeb-4cae-8912-604df35afc71-config-out\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.662782 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662780 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57f53419-1aeb-4cae-8912-604df35afc71-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.663015 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.663015 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.663015 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.662869 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-config-volume\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.763931 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.763880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57f53419-1aeb-4cae-8912-604df35afc71-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.763931 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.763939 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57f53419-1aeb-4cae-8912-604df35afc71-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764070 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.763982 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg9r4\" (UniqueName: \"kubernetes.io/projected/57f53419-1aeb-4cae-8912-604df35afc71-kube-api-access-gg9r4\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764070 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764018 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57f53419-1aeb-4cae-8912-604df35afc71-config-out\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764070 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764044 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57f53419-1aeb-4cae-8912-604df35afc71-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764165 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764083 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764165 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764101 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764165 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764137 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-config-volume\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764291 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764175 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-web-config\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764291 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764204 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764291 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764228 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764291 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764253 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764291 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764278 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57f53419-1aeb-4cae-8912-604df35afc71-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764651 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764598 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57f53419-1aeb-4cae-8912-604df35afc71-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.764913 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.764863 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57f53419-1aeb-4cae-8912-604df35afc71-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.765218 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.765191 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57f53419-1aeb-4cae-8912-604df35afc71-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.767208 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.767097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57f53419-1aeb-4cae-8912-604df35afc71-config-out\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.767208 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.767191 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57f53419-1aeb-4cae-8912-604df35afc71-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.767552 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.767509 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.767785 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.767757 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.767877 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.767761 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.767877 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.767869 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.767986 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.767952 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.767986 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.767954 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-web-config\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.769205 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.769184 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57f53419-1aeb-4cae-8912-604df35afc71-config-volume\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.771279 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.771229 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg9r4\" (UniqueName: \"kubernetes.io/projected/57f53419-1aeb-4cae-8912-604df35afc71-kube-api-access-gg9r4\") pod \"alertmanager-main-0\" (UID: \"57f53419-1aeb-4cae-8912-604df35afc71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:21.878096 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:21.878038 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 31 19:29:22.018593 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:22.018567 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 31 19:29:22.021254 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:29:22.021218 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f53419_1aeb_4cae_8912_604df35afc71.slice/crio-647488b63758a06bc5ecc1e068bc4b1db9fca37686a4daf4c95fb2b8cc66756e WatchSource:0}: Error finding container 647488b63758a06bc5ecc1e068bc4b1db9fca37686a4daf4c95fb2b8cc66756e: Status 404 returned error can't find the container with id 647488b63758a06bc5ecc1e068bc4b1db9fca37686a4daf4c95fb2b8cc66756e Jul 31 19:29:22.519199 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:22.519161 2565 generic.go:356] "Generic (PLEG): container finished" podID="57f53419-1aeb-4cae-8912-604df35afc71" containerID="96b74962c4400955973a48fe38e29367c90d806800b6022d9bc1a71f2e2b16d2" exitCode=0 Jul 31 19:29:22.519614 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:22.519256 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57f53419-1aeb-4cae-8912-604df35afc71","Type":"ContainerDied","Data":"96b74962c4400955973a48fe38e29367c90d806800b6022d9bc1a71f2e2b16d2"} Jul 31 19:29:22.519614 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:22.519312 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57f53419-1aeb-4cae-8912-604df35afc71","Type":"ContainerStarted","Data":"647488b63758a06bc5ecc1e068bc4b1db9fca37686a4daf4c95fb2b8cc66756e"} Jul 31 19:29:23.263077 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:23.263043 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5103512d-815c-40d8-805e-af4c7f6be3ea" path="/var/lib/kubelet/pods/5103512d-815c-40d8-805e-af4c7f6be3ea/volumes" Jul 31 19:29:23.526581 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:23.526494 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57f53419-1aeb-4cae-8912-604df35afc71","Type":"ContainerStarted","Data":"8b6ac8c5cce4a3f71d86e2ab981698981589ca74e127b068043849d54fb2168c"} Jul 31 19:29:23.526581 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:23.526570 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57f53419-1aeb-4cae-8912-604df35afc71","Type":"ContainerStarted","Data":"d1c5c8467a513eb9b6764b7a4f8832a4868eb298c257e0bb9bf40f3ac4c2e922"} Jul 31 19:29:23.527119 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:23.526608 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57f53419-1aeb-4cae-8912-604df35afc71","Type":"ContainerStarted","Data":"c8fa85f262eeea990970effd399bbf6ed6aee0fe12957318cb1a23810fc80fad"} Jul 31 19:29:23.527119 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:23.526651 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57f53419-1aeb-4cae-8912-604df35afc71","Type":"ContainerStarted","Data":"930cf6369fdebe2701e5c906e8b3233e7ee9e39ed110613c482cdf4da68bde2a"} Jul 31 19:29:23.527119 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:23.526690 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57f53419-1aeb-4cae-8912-604df35afc71","Type":"ContainerStarted","Data":"48350a2ce4a9dcb878bb0893ea3425920395715bef05735f8a36102d659d73e1"} Jul 31 19:29:23.527119 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:23.526737 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57f53419-1aeb-4cae-8912-604df35afc71","Type":"ContainerStarted","Data":"ff26f8a48005a01fb7e7e5135c5a936564a098b2628cf513a4feecac608b34e5"} Jul 31 19:29:23.555295 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:29:23.555239 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.540791398 podStartE2EDuration="2.555213361s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.421963ms" isStatefulPod=false podCreationTimestamp="2026-07-31 19:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:29:23.554175958 +0000 UTC m=+358.825523335" watchObservedRunningTime="2026-07-31 19:29:23.555213361 +0000 UTC m=+358.826560714" Jul 31 19:30:20.440619 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.440586 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p"] Jul 31 19:30:20.444250 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.444230 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.446439 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.446418 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 31 19:30:20.446574 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.446418 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 31 19:30:20.446574 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.446471 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 31 19:30:20.447260 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.447242 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 31 19:30:20.447368 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.447312 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 31 19:30:20.447368 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.447250 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 31 19:30:20.447368 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.447315 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 31 19:30:20.453428 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.453410 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p"] Jul 31 19:30:20.530429 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.530393 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-ca\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.530429 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.530427 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.530629 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.530457 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj5n8\" (UniqueName: \"kubernetes.io/projected/b77ba9a7-096f-42fd-9ff8-c042d326b36f-kube-api-access-bj5n8\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.530629 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.530535 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b77ba9a7-096f-42fd-9ff8-c042d326b36f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.530629 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.530594 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-hub\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.530629 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.530621 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.631756 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.631724 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b77ba9a7-096f-42fd-9ff8-c042d326b36f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.631890 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.631770 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-hub\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.631890 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.631795 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.631890 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.631835 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-ca\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.631890 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.631858 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.632050 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.631896 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj5n8\" (UniqueName: \"kubernetes.io/projected/b77ba9a7-096f-42fd-9ff8-c042d326b36f-kube-api-access-bj5n8\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.632503 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.632476 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b77ba9a7-096f-42fd-9ff8-c042d326b36f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.634371 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.634327 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-ca\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.634538 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.634520 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.634694 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.634679 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-hub\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.634790 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.634771 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b77ba9a7-096f-42fd-9ff8-c042d326b36f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.638406 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.638389 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj5n8\" (UniqueName: \"kubernetes.io/projected/b77ba9a7-096f-42fd-9ff8-c042d326b36f-kube-api-access-bj5n8\") pod \"cluster-proxy-proxy-agent-7c8f54d856-c9k5p\" (UID: \"b77ba9a7-096f-42fd-9ff8-c042d326b36f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.766891 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.766811 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" Jul 31 19:30:20.897993 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.897910 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p"] Jul 31 19:30:20.900576 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:30:20.900541 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb77ba9a7_096f_42fd_9ff8_c042d326b36f.slice/crio-75bf6c0acf26e603d21f53f2a9f7ad4a6cab7f90a28d03f46735cfbe54fe6767 WatchSource:0}: Error finding container 75bf6c0acf26e603d21f53f2a9f7ad4a6cab7f90a28d03f46735cfbe54fe6767: Status 404 returned error can't find the container with id 75bf6c0acf26e603d21f53f2a9f7ad4a6cab7f90a28d03f46735cfbe54fe6767 Jul 31 19:30:20.902163 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:20.902147 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 19:30:21.755730 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:21.755696 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" event={"ID":"b77ba9a7-096f-42fd-9ff8-c042d326b36f","Type":"ContainerStarted","Data":"75bf6c0acf26e603d21f53f2a9f7ad4a6cab7f90a28d03f46735cfbe54fe6767"} Jul 31 19:30:24.775206 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:24.775182 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" event={"ID":"b77ba9a7-096f-42fd-9ff8-c042d326b36f","Type":"ContainerStarted","Data":"e51ef639d0ad9c60e910c91e58b256dcf7f6ac36169eecfa15102dfffdab2eba"} Jul 31 19:30:25.780566 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:25.780534 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" event={"ID":"b77ba9a7-096f-42fd-9ff8-c042d326b36f","Type":"ContainerStarted","Data":"418c65dff3ebc8e7edadd2b99cd59e53b284e6411aa3914dfc6d948438b66fc9"} Jul 31 19:30:25.780944 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:25.780575 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" event={"ID":"b77ba9a7-096f-42fd-9ff8-c042d326b36f","Type":"ContainerStarted","Data":"3ef5f6919f0c3e9800ee6320ef1d96a970ea38f4f0d03b54e16fd6f0ba202d0e"} Jul 31 19:30:25.797109 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:30:25.797057 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7c8f54d856-c9k5p" podStartSLOduration=2.044710467 podStartE2EDuration="5.79704105s" totalImagesPullingTime="3.752330583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:30:25.795756951 +0000 UTC m=+421.067104305" watchObservedRunningTime="2026-07-31 19:30:25.79704105 +0000 UTC m=+421.068388426" Jul 31 19:31:48.185036 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.185000 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k"] Jul 31 19:31:48.188291 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.188272 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.190558 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.190489 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 31 19:31:48.190558 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.190522 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 31 19:31:48.190747 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.190687 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 31 19:31:48.191205 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.191180 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zptlg\"" Jul 31 19:31:48.200988 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.200939 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 31 19:31:48.206400 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.206369 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k"] Jul 31 19:31:48.254718 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.254683 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43c17712-de70-4e6f-81ad-45a33d71cefe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.254718 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.254720 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6kpf\" (UniqueName: \"kubernetes.io/projected/43c17712-de70-4e6f-81ad-45a33d71cefe-kube-api-access-z6kpf\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.254968 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.254755 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43c17712-de70-4e6f-81ad-45a33d71cefe-tmp\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.254968 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.254810 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/43c17712-de70-4e6f-81ad-45a33d71cefe-manifests\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.254968 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.254829 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43c17712-de70-4e6f-81ad-45a33d71cefe-webhook-cert\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.355359 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.355310 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43c17712-de70-4e6f-81ad-45a33d71cefe-tmp\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.355536 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.355405 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/43c17712-de70-4e6f-81ad-45a33d71cefe-manifests\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.355536 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.355434 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43c17712-de70-4e6f-81ad-45a33d71cefe-webhook-cert\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.355536 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.355524 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43c17712-de70-4e6f-81ad-45a33d71cefe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.355668 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.355549 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6kpf\" (UniqueName: \"kubernetes.io/projected/43c17712-de70-4e6f-81ad-45a33d71cefe-kube-api-access-z6kpf\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.355940 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.355917 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/43c17712-de70-4e6f-81ad-45a33d71cefe-manifests\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.357886 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.357851 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43c17712-de70-4e6f-81ad-45a33d71cefe-tmp\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.358060 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.358041 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43c17712-de70-4e6f-81ad-45a33d71cefe-webhook-cert\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.358121 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.358097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43c17712-de70-4e6f-81ad-45a33d71cefe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.364901 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.364875 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6kpf\" (UniqueName: \"kubernetes.io/projected/43c17712-de70-4e6f-81ad-45a33d71cefe-kube-api-access-z6kpf\") pod \"opendatahub-operator-controller-manager-bbd966b49-8l77k\" (UID: \"43c17712-de70-4e6f-81ad-45a33d71cefe\") " pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.499815 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.499722 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:48.647878 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:48.647841 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k"] Jul 31 19:31:48.651467 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:31:48.651440 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c17712_de70_4e6f_81ad_45a33d71cefe.slice/crio-e3e0231eb005a8ccbd0ba12f4a4d31c27431cb53c47342800a3497ca1fb7cd8c WatchSource:0}: Error finding container e3e0231eb005a8ccbd0ba12f4a4d31c27431cb53c47342800a3497ca1fb7cd8c: Status 404 returned error can't find the container with id e3e0231eb005a8ccbd0ba12f4a4d31c27431cb53c47342800a3497ca1fb7cd8c Jul 31 19:31:49.109772 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:49.109731 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" event={"ID":"43c17712-de70-4e6f-81ad-45a33d71cefe","Type":"ContainerStarted","Data":"e3e0231eb005a8ccbd0ba12f4a4d31c27431cb53c47342800a3497ca1fb7cd8c"} Jul 31 19:31:52.127314 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:52.127274 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" event={"ID":"43c17712-de70-4e6f-81ad-45a33d71cefe","Type":"ContainerStarted","Data":"282232d045e8f4bdd7abc4365c1cd1da40893a8094547bb9554c97984bd28a06"} Jul 31 19:31:53.133023 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:53.132987 2565 generic.go:356] "Generic (PLEG): container finished" podID="43c17712-de70-4e6f-81ad-45a33d71cefe" containerID="282232d045e8f4bdd7abc4365c1cd1da40893a8094547bb9554c97984bd28a06" exitCode=0 Jul 31 19:31:53.133455 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:53.133036 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" event={"ID":"43c17712-de70-4e6f-81ad-45a33d71cefe","Type":"ContainerDied","Data":"282232d045e8f4bdd7abc4365c1cd1da40893a8094547bb9554c97984bd28a06"} Jul 31 19:31:54.140084 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:54.140047 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" event={"ID":"43c17712-de70-4e6f-81ad-45a33d71cefe","Type":"ContainerStarted","Data":"f14d0d6ec2ad2591c2cf76dda759bc0c9e7db9ad9f3cb8084ae422f85686c6f2"} Jul 31 19:31:54.140494 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:54.140184 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:31:54.159885 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:31:54.159834 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" podStartSLOduration=1.618189749 podStartE2EDuration="6.159811995s" totalImagesPullingTime="3.482058714s" totalInitContainerRuntime="1.059563532s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:31:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:31:54.157977585 +0000 UTC m=+509.429324940" watchObservedRunningTime="2026-07-31 19:31:54.159811995 +0000 UTC m=+509.431159349" Jul 31 19:32:05.146148 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:05.146119 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bbd966b49-8l77k" Jul 31 19:32:07.959160 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:07.959121 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch"] Jul 31 19:32:07.968513 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:07.968485 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" Jul 31 19:32:07.970730 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:07.970695 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 31 19:32:07.970730 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:07.970720 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 31 19:32:07.971626 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:07.971601 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch"] Jul 31 19:32:08.014611 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.014551 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7957d167-131d-46ca-b30f-9731ebb19684-tmp\") pod \"kube-auth-proxy-6bcff68c6d-bspch\" (UID: \"7957d167-131d-46ca-b30f-9731ebb19684\") " pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" Jul 31 19:32:08.014820 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.014641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7957d167-131d-46ca-b30f-9731ebb19684-tls-certs\") pod \"kube-auth-proxy-6bcff68c6d-bspch\" (UID: \"7957d167-131d-46ca-b30f-9731ebb19684\") " pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" Jul 31 19:32:08.014820 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.014659 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqvgp\" (UniqueName: \"kubernetes.io/projected/7957d167-131d-46ca-b30f-9731ebb19684-kube-api-access-dqvgp\") pod \"kube-auth-proxy-6bcff68c6d-bspch\" (UID: \"7957d167-131d-46ca-b30f-9731ebb19684\") " pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" Jul 31 19:32:08.115273 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.115239 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7957d167-131d-46ca-b30f-9731ebb19684-tmp\") pod \"kube-auth-proxy-6bcff68c6d-bspch\" (UID: \"7957d167-131d-46ca-b30f-9731ebb19684\") " pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" Jul 31 19:32:08.115464 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.115289 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7957d167-131d-46ca-b30f-9731ebb19684-tls-certs\") pod \"kube-auth-proxy-6bcff68c6d-bspch\" (UID: \"7957d167-131d-46ca-b30f-9731ebb19684\") " pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" Jul 31 19:32:08.115464 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.115408 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqvgp\" (UniqueName: \"kubernetes.io/projected/7957d167-131d-46ca-b30f-9731ebb19684-kube-api-access-dqvgp\") pod \"kube-auth-proxy-6bcff68c6d-bspch\" (UID: \"7957d167-131d-46ca-b30f-9731ebb19684\") " pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" Jul 31 19:32:08.117774 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.117746 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7957d167-131d-46ca-b30f-9731ebb19684-tmp\") pod \"kube-auth-proxy-6bcff68c6d-bspch\" (UID: \"7957d167-131d-46ca-b30f-9731ebb19684\") " pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" Jul 31 19:32:08.118174 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.118150 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7957d167-131d-46ca-b30f-9731ebb19684-tls-certs\") pod \"kube-auth-proxy-6bcff68c6d-bspch\" (UID: \"7957d167-131d-46ca-b30f-9731ebb19684\") " pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" Jul 31 19:32:08.128389 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.128333 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqvgp\" (UniqueName: \"kubernetes.io/projected/7957d167-131d-46ca-b30f-9731ebb19684-kube-api-access-dqvgp\") pod \"kube-auth-proxy-6bcff68c6d-bspch\" (UID: \"7957d167-131d-46ca-b30f-9731ebb19684\") " pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" Jul 31 19:32:08.182122 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.182081 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kqxl5"] Jul 31 19:32:08.184906 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.184879 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" Jul 31 19:32:08.187672 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.187651 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-gdln9\"" Jul 31 19:32:08.187934 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.187915 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 31 19:32:08.198746 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.198715 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kqxl5"] Jul 31 19:32:08.216176 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.216081 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6pkb\" (UniqueName: \"kubernetes.io/projected/9830454e-4c7a-4be4-ade8-5ee6caabae0f-kube-api-access-f6pkb\") pod \"odh-model-controller-7998c67dc4-kqxl5\" (UID: \"9830454e-4c7a-4be4-ade8-5ee6caabae0f\") " pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" Jul 31 19:32:08.216176 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.216148 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9830454e-4c7a-4be4-ade8-5ee6caabae0f-cert\") pod \"odh-model-controller-7998c67dc4-kqxl5\" (UID: \"9830454e-4c7a-4be4-ade8-5ee6caabae0f\") " pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" Jul 31 19:32:08.280315 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.280278 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" Jul 31 19:32:08.316763 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.316724 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6pkb\" (UniqueName: \"kubernetes.io/projected/9830454e-4c7a-4be4-ade8-5ee6caabae0f-kube-api-access-f6pkb\") pod \"odh-model-controller-7998c67dc4-kqxl5\" (UID: \"9830454e-4c7a-4be4-ade8-5ee6caabae0f\") " pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" Jul 31 19:32:08.316948 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.316783 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9830454e-4c7a-4be4-ade8-5ee6caabae0f-cert\") pod \"odh-model-controller-7998c67dc4-kqxl5\" (UID: \"9830454e-4c7a-4be4-ade8-5ee6caabae0f\") " pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" Jul 31 19:32:08.316948 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:32:08.316927 2565 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 31 19:32:08.317038 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:32:08.317009 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9830454e-4c7a-4be4-ade8-5ee6caabae0f-cert podName:9830454e-4c7a-4be4-ade8-5ee6caabae0f nodeName:}" failed. No retries permitted until 2026-07-31 19:32:08.81698856 +0000 UTC m=+524.088335906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9830454e-4c7a-4be4-ade8-5ee6caabae0f-cert") pod "odh-model-controller-7998c67dc4-kqxl5" (UID: "9830454e-4c7a-4be4-ade8-5ee6caabae0f") : secret "odh-model-controller-webhook-cert" not found Jul 31 19:32:08.331258 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.331193 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6pkb\" (UniqueName: \"kubernetes.io/projected/9830454e-4c7a-4be4-ade8-5ee6caabae0f-kube-api-access-f6pkb\") pod \"odh-model-controller-7998c67dc4-kqxl5\" (UID: \"9830454e-4c7a-4be4-ade8-5ee6caabae0f\") " pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" Jul 31 19:32:08.436460 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.436425 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch"] Jul 31 19:32:08.439774 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:32:08.439745 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7957d167_131d_46ca_b30f_9731ebb19684.slice/crio-acccd98065ad61496f180c43e34f65a8974adbaf58aeb5a11e8c49a18d3a827e WatchSource:0}: Error finding container acccd98065ad61496f180c43e34f65a8974adbaf58aeb5a11e8c49a18d3a827e: Status 404 returned error can't find the container with id acccd98065ad61496f180c43e34f65a8974adbaf58aeb5a11e8c49a18d3a827e Jul 31 19:32:08.819074 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.819034 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9830454e-4c7a-4be4-ade8-5ee6caabae0f-cert\") pod \"odh-model-controller-7998c67dc4-kqxl5\" (UID: \"9830454e-4c7a-4be4-ade8-5ee6caabae0f\") " pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" Jul 31 19:32:08.821885 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:08.821859 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9830454e-4c7a-4be4-ade8-5ee6caabae0f-cert\") pod \"odh-model-controller-7998c67dc4-kqxl5\" (UID: \"9830454e-4c7a-4be4-ade8-5ee6caabae0f\") " pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" Jul 31 19:32:09.096574 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.096472 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" Jul 31 19:32:09.110927 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.110888 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-krdgh"] Jul 31 19:32:09.113912 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.113887 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" Jul 31 19:32:09.116234 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.116195 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 31 19:32:09.116398 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.116245 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-bgrmm\"" Jul 31 19:32:09.121910 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.121867 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l24mw\" (UniqueName: \"kubernetes.io/projected/75366a00-6e93-450b-bb3f-54bd8dd9e195-kube-api-access-l24mw\") pod \"model-serving-api-54d89bd5fc-krdgh\" (UID: \"75366a00-6e93-450b-bb3f-54bd8dd9e195\") " pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" Jul 31 19:32:09.122054 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.121973 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75366a00-6e93-450b-bb3f-54bd8dd9e195-tls-certs\") pod \"model-serving-api-54d89bd5fc-krdgh\" (UID: \"75366a00-6e93-450b-bb3f-54bd8dd9e195\") " pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" Jul 31 19:32:09.122513 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.122482 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-krdgh"] Jul 31 19:32:09.204987 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.204934 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" event={"ID":"7957d167-131d-46ca-b30f-9731ebb19684","Type":"ContainerStarted","Data":"acccd98065ad61496f180c43e34f65a8974adbaf58aeb5a11e8c49a18d3a827e"} Jul 31 19:32:09.223376 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.223274 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75366a00-6e93-450b-bb3f-54bd8dd9e195-tls-certs\") pod \"model-serving-api-54d89bd5fc-krdgh\" (UID: \"75366a00-6e93-450b-bb3f-54bd8dd9e195\") " pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" Jul 31 19:32:09.223376 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.223371 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l24mw\" (UniqueName: \"kubernetes.io/projected/75366a00-6e93-450b-bb3f-54bd8dd9e195-kube-api-access-l24mw\") pod \"model-serving-api-54d89bd5fc-krdgh\" (UID: \"75366a00-6e93-450b-bb3f-54bd8dd9e195\") " pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" Jul 31 19:32:09.227296 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.227265 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75366a00-6e93-450b-bb3f-54bd8dd9e195-tls-certs\") pod \"model-serving-api-54d89bd5fc-krdgh\" (UID: \"75366a00-6e93-450b-bb3f-54bd8dd9e195\") " pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" Jul 31 19:32:09.232139 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.232097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l24mw\" (UniqueName: \"kubernetes.io/projected/75366a00-6e93-450b-bb3f-54bd8dd9e195-kube-api-access-l24mw\") pod \"model-serving-api-54d89bd5fc-krdgh\" (UID: \"75366a00-6e93-450b-bb3f-54bd8dd9e195\") " pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" Jul 31 19:32:09.254056 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.254016 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kqxl5"] Jul 31 19:32:09.256384 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.256363 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd"] Jul 31 19:32:09.258130 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:32:09.258100 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9830454e_4c7a_4be4_ade8_5ee6caabae0f.slice/crio-922a2d9a0a67f8a04c14d0ffdbea0d33e07abd386593d285674780797f77c870 WatchSource:0}: Error finding container 922a2d9a0a67f8a04c14d0ffdbea0d33e07abd386593d285674780797f77c870: Status 404 returned error can't find the container with id 922a2d9a0a67f8a04c14d0ffdbea0d33e07abd386593d285674780797f77c870 Jul 31 19:32:09.259461 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.259437 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.261533 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.261511 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 31 19:32:09.261657 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.261570 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 31 19:32:09.261657 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.261589 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v5dqr\"" Jul 31 19:32:09.261877 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.261864 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 31 19:32:09.261976 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.261885 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 31 19:32:09.273472 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.273445 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd"] Jul 31 19:32:09.324739 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.324707 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.324739 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.324741 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1835f4b8-db03-4005-99df-bbc9406dd239-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.324975 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.324762 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w2zt\" (UniqueName: \"kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-kube-api-access-2w2zt\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.324975 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.324793 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.324975 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.324864 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.324975 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.324929 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.325140 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.324982 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.425517 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.425416 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.425517 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.425481 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.425752 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.425537 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.425752 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.425559 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1835f4b8-db03-4005-99df-bbc9406dd239-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.425752 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.425600 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w2zt\" (UniqueName: \"kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-kube-api-access-2w2zt\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.425752 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.425638 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.425752 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.425675 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.426237 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.426211 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.428076 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.428045 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.428198 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.428096 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1835f4b8-db03-4005-99df-bbc9406dd239-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.428198 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.428172 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.428413 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.428395 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.433415 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.433325 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w2zt\" (UniqueName: \"kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-kube-api-access-2w2zt\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.433620 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.433599 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-49kjd\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.443486 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.443458 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" Jul 31 19:32:09.575785 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.575736 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:09.614139 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.614054 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-krdgh"] Jul 31 19:32:09.751783 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:09.751731 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd"] Jul 31 19:32:09.754186 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:32:09.754158 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1835f4b8_db03_4005_99df_bbc9406dd239.slice/crio-445251782e6f6bc63d675786af1562e00850e030b9fa93023ee5d7b1fdda55ca WatchSource:0}: Error finding container 445251782e6f6bc63d675786af1562e00850e030b9fa93023ee5d7b1fdda55ca: Status 404 returned error can't find the container with id 445251782e6f6bc63d675786af1562e00850e030b9fa93023ee5d7b1fdda55ca Jul 31 19:32:10.212337 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:10.212293 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" event={"ID":"1835f4b8-db03-4005-99df-bbc9406dd239","Type":"ContainerStarted","Data":"445251782e6f6bc63d675786af1562e00850e030b9fa93023ee5d7b1fdda55ca"} Jul 31 19:32:10.222795 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:10.220203 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" event={"ID":"75366a00-6e93-450b-bb3f-54bd8dd9e195","Type":"ContainerStarted","Data":"cc367126769eed64f77bc1c6fb18124186443ad8e659e3697ca6cfe82e861fbc"} Jul 31 19:32:10.226862 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:10.226819 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" event={"ID":"9830454e-4c7a-4be4-ade8-5ee6caabae0f","Type":"ContainerStarted","Data":"922a2d9a0a67f8a04c14d0ffdbea0d33e07abd386593d285674780797f77c870"} Jul 31 19:32:14.868960 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:14.868918 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 31 19:32:14.869358 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:14.869037 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 31 19:32:15.268119 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.268079 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" Jul 31 19:32:15.268421 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.268143 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" Jul 31 19:32:15.268421 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.268166 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" event={"ID":"7957d167-131d-46ca-b30f-9731ebb19684","Type":"ContainerStarted","Data":"c55b3c9b943af9a96e48e725a8788652e0429795096516574080e6663da8590e"} Jul 31 19:32:15.268421 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.268195 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" event={"ID":"75366a00-6e93-450b-bb3f-54bd8dd9e195","Type":"ContainerStarted","Data":"bbbefe364b952b64d91b8826f34318b14c2a0779d0a8238ba58f3b575e111f34"} Jul 31 19:32:15.268421 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.268220 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" event={"ID":"9830454e-4c7a-4be4-ade8-5ee6caabae0f","Type":"ContainerStarted","Data":"af907515deae7dd3d5f3105c93e591f779383b4267cb55fada0e0ca22f601c34"} Jul 31 19:32:15.270175 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.270094 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" event={"ID":"1835f4b8-db03-4005-99df-bbc9406dd239","Type":"ContainerStarted","Data":"2376b02850af0ce4b03ce33f36e6709847736effaa65efa310b08e27d15fa703"} Jul 31 19:32:15.270300 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.270247 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:15.272401 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.272369 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-49kjd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 19:32:15.272530 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.272429 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" podUID="1835f4b8-db03-4005-99df-bbc9406dd239" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 19:32:15.281122 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.281075 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bcff68c6d-bspch" podStartSLOduration=1.8696419610000001 podStartE2EDuration="8.281060509s" totalImagesPullingTime="6.411418548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:32:15.279934227 +0000 UTC m=+530.551281583" watchObservedRunningTime="2026-07-31 19:32:15.281060509 +0000 UTC m=+530.552407863" Jul 31 19:32:15.297423 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.297327 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" podStartSLOduration=1.689781186 podStartE2EDuration="7.297305145s" totalImagesPullingTime="5.607523959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:32:15.293997711 +0000 UTC m=+530.565345066" watchObservedRunningTime="2026-07-31 19:32:15.297305145 +0000 UTC m=+530.568652501" Jul 31 19:32:15.333107 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.333046 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" podStartSLOduration=1.098428821 podStartE2EDuration="6.333022092s" totalImagesPullingTime="5.234593271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:32:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:32:15.313216378 +0000 UTC m=+530.584563734" watchObservedRunningTime="2026-07-31 19:32:15.333022092 +0000 UTC m=+530.604369448" Jul 31 19:32:15.335565 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:15.335510 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" podStartSLOduration=1.223714463 podStartE2EDuration="6.33549556s" totalImagesPullingTime="5.111781097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:32:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:32:15.33337164 +0000 UTC m=+530.604718980" watchObservedRunningTime="2026-07-31 19:32:15.33549556 +0000 UTC m=+530.606842917" Jul 31 19:32:16.275943 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.275868 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-49kjd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 19:32:16.275943 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.275914 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" podUID="1835f4b8-db03-4005-99df-bbc9406dd239" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 19:32:16.400912 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.400874 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6"] Jul 31 19:32:16.405088 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.405063 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.408418 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.408392 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-8bpkg\"" Jul 31 19:32:16.422857 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.422827 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6"] Jul 31 19:32:16.485843 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.485765 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.486121 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.486094 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.486237 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.486127 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.486237 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.486165 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.486237 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.486217 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.486394 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.486246 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.486394 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.486269 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdrgg\" (UniqueName: \"kubernetes.io/projected/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-kube-api-access-fdrgg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.486394 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.486333 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.486394 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.486393 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.587134 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.587095 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.587509 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.587145 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.587509 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.587168 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.587509 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.587207 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.587509 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.587258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.587509 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.587292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.587509 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.587321 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdrgg\" (UniqueName: \"kubernetes.io/projected/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-kube-api-access-fdrgg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.587509 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.587410 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.587509 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.587439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.588329 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.588303 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.589510 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.589480 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.589625 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.588843 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.589701 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.588793 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.589866 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.589793 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.595413 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.592867 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.601023 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.598991 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.601023 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.599737 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdrgg\" (UniqueName: \"kubernetes.io/projected/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-kube-api-access-fdrgg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.602710 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.602686 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5923e4f8-49e1-4a4e-91ac-0cc9023eb723-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6\" (UID: \"5923e4f8-49e1-4a4e-91ac-0cc9023eb723\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.716587 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.716544 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:16.881466 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:16.881434 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6"] Jul 31 19:32:16.883107 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:32:16.883075 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5923e4f8_49e1_4a4e_91ac_0cc9023eb723.slice/crio-f6cba1166cf9ca573678103bcb067d1904be25731921da78cc21f0dfb53df960 WatchSource:0}: Error finding container f6cba1166cf9ca573678103bcb067d1904be25731921da78cc21f0dfb53df960: Status 404 returned error can't find the container with id f6cba1166cf9ca573678103bcb067d1904be25731921da78cc21f0dfb53df960 Jul 31 19:32:17.280453 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:17.280363 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" event={"ID":"5923e4f8-49e1-4a4e-91ac-0cc9023eb723","Type":"ContainerStarted","Data":"f6cba1166cf9ca573678103bcb067d1904be25731921da78cc21f0dfb53df960"} Jul 31 19:32:19.276077 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.276045 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:19.465778 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.465726 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nshht"] Jul 31 19:32:19.477370 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.477329 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n"] Jul 31 19:32:19.477542 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.477453 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" Jul 31 19:32:19.479814 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.479790 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 31 19:32:19.479961 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.479796 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-7c86l\"" Jul 31 19:32:19.481224 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.481174 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nshht"] Jul 31 19:32:19.481383 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.481262 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" Jul 31 19:32:19.483241 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.483177 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 31 19:32:19.483241 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.483197 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-4tmrx\"" Jul 31 19:32:19.501168 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.500905 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n"] Jul 31 19:32:19.515088 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.515058 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3ee0c60-5559-42b2-93a6-069acf139e49-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jcw7n\" (UID: \"c3ee0c60-5559-42b2-93a6-069acf139e49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" Jul 31 19:32:19.515279 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.515101 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b2f107d-07cd-4d05-a7f3-c40d3c30162b-cert\") pod \"kserve-controller-manager-d6f4b65b8-nshht\" (UID: \"1b2f107d-07cd-4d05-a7f3-c40d3c30162b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" Jul 31 19:32:19.515279 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.515129 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7tff\" (UniqueName: \"kubernetes.io/projected/c3ee0c60-5559-42b2-93a6-069acf139e49-kube-api-access-v7tff\") pod \"llmisvc-controller-manager-777c6bd7d5-jcw7n\" (UID: \"c3ee0c60-5559-42b2-93a6-069acf139e49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" Jul 31 19:32:19.515279 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.515259 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5hpj\" (UniqueName: \"kubernetes.io/projected/1b2f107d-07cd-4d05-a7f3-c40d3c30162b-kube-api-access-f5hpj\") pod \"kserve-controller-manager-d6f4b65b8-nshht\" (UID: \"1b2f107d-07cd-4d05-a7f3-c40d3c30162b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" Jul 31 19:32:19.598558 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.598520 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 31 19:32:19.598685 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.598594 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 31 19:32:19.598685 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.598635 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 31 19:32:19.617040 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.615963 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3ee0c60-5559-42b2-93a6-069acf139e49-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jcw7n\" (UID: \"c3ee0c60-5559-42b2-93a6-069acf139e49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" Jul 31 19:32:19.617040 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.616021 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b2f107d-07cd-4d05-a7f3-c40d3c30162b-cert\") pod \"kserve-controller-manager-d6f4b65b8-nshht\" (UID: \"1b2f107d-07cd-4d05-a7f3-c40d3c30162b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" Jul 31 19:32:19.617040 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.616048 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7tff\" (UniqueName: \"kubernetes.io/projected/c3ee0c60-5559-42b2-93a6-069acf139e49-kube-api-access-v7tff\") pod \"llmisvc-controller-manager-777c6bd7d5-jcw7n\" (UID: \"c3ee0c60-5559-42b2-93a6-069acf139e49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" Jul 31 19:32:19.617040 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.616117 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5hpj\" (UniqueName: \"kubernetes.io/projected/1b2f107d-07cd-4d05-a7f3-c40d3c30162b-kube-api-access-f5hpj\") pod \"kserve-controller-manager-d6f4b65b8-nshht\" (UID: \"1b2f107d-07cd-4d05-a7f3-c40d3c30162b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" Jul 31 19:32:19.617040 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:32:19.616639 2565 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 31 19:32:19.617040 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:32:19.616703 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3ee0c60-5559-42b2-93a6-069acf139e49-cert podName:c3ee0c60-5559-42b2-93a6-069acf139e49 nodeName:}" failed. No retries permitted until 2026-07-31 19:32:20.116682518 +0000 UTC m=+535.388029853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c3ee0c60-5559-42b2-93a6-069acf139e49-cert") pod "llmisvc-controller-manager-777c6bd7d5-jcw7n" (UID: "c3ee0c60-5559-42b2-93a6-069acf139e49") : secret "llmisvc-webhook-server-cert" not found Jul 31 19:32:19.620679 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.620651 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b2f107d-07cd-4d05-a7f3-c40d3c30162b-cert\") pod \"kserve-controller-manager-d6f4b65b8-nshht\" (UID: \"1b2f107d-07cd-4d05-a7f3-c40d3c30162b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" Jul 31 19:32:19.636132 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.636089 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7tff\" (UniqueName: \"kubernetes.io/projected/c3ee0c60-5559-42b2-93a6-069acf139e49-kube-api-access-v7tff\") pod \"llmisvc-controller-manager-777c6bd7d5-jcw7n\" (UID: \"c3ee0c60-5559-42b2-93a6-069acf139e49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" Jul 31 19:32:19.637639 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.637606 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5hpj\" (UniqueName: \"kubernetes.io/projected/1b2f107d-07cd-4d05-a7f3-c40d3c30162b-kube-api-access-f5hpj\") pod \"kserve-controller-manager-d6f4b65b8-nshht\" (UID: \"1b2f107d-07cd-4d05-a7f3-c40d3c30162b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" Jul 31 19:32:19.796392 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.796331 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" Jul 31 19:32:19.978883 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:19.978855 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nshht"] Jul 31 19:32:19.981276 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:32:19.981238 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b2f107d_07cd_4d05_a7f3_c40d3c30162b.slice/crio-f08ee7710633788d454da98d736e3a98feb188fca459b4364d2ebe6570da1b8d WatchSource:0}: Error finding container f08ee7710633788d454da98d736e3a98feb188fca459b4364d2ebe6570da1b8d: Status 404 returned error can't find the container with id f08ee7710633788d454da98d736e3a98feb188fca459b4364d2ebe6570da1b8d Jul 31 19:32:20.120964 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:20.120925 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3ee0c60-5559-42b2-93a6-069acf139e49-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jcw7n\" (UID: \"c3ee0c60-5559-42b2-93a6-069acf139e49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" Jul 31 19:32:20.123483 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:20.123465 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3ee0c60-5559-42b2-93a6-069acf139e49-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jcw7n\" (UID: \"c3ee0c60-5559-42b2-93a6-069acf139e49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" Jul 31 19:32:20.302109 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:20.302024 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" event={"ID":"1b2f107d-07cd-4d05-a7f3-c40d3c30162b","Type":"ContainerStarted","Data":"f08ee7710633788d454da98d736e3a98feb188fca459b4364d2ebe6570da1b8d"} Jul 31 19:32:20.303622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:20.303599 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" event={"ID":"5923e4f8-49e1-4a4e-91ac-0cc9023eb723","Type":"ContainerStarted","Data":"d9ce28d6a02aea643adc7fcdce4025c0cd813389fae61d2368a776ec80043b1d"} Jul 31 19:32:20.327927 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:20.327880 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" podStartSLOduration=1.6153490000000001 podStartE2EDuration="4.327864439s" totalImagesPullingTime="2.712515439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:32:20.326630156 +0000 UTC m=+535.597977510" watchObservedRunningTime="2026-07-31 19:32:20.327864439 +0000 UTC m=+535.599211793" Jul 31 19:32:20.406180 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:20.406144 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" Jul 31 19:32:20.557459 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:20.557382 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n"] Jul 31 19:32:20.560523 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:32:20.560499 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc3ee0c60_5559_42b2_93a6_069acf139e49.slice/crio-f64d258b7ba88dce5bf35ba3e94c436a5951bd917480c37da6936a8cd49c8323 WatchSource:0}: Error finding container f64d258b7ba88dce5bf35ba3e94c436a5951bd917480c37da6936a8cd49c8323: Status 404 returned error can't find the container with id f64d258b7ba88dce5bf35ba3e94c436a5951bd917480c37da6936a8cd49c8323 Jul 31 19:32:20.717733 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:20.717696 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:20.722853 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:20.722821 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:21.311014 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:21.310968 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" event={"ID":"c3ee0c60-5559-42b2-93a6-069acf139e49","Type":"ContainerStarted","Data":"f64d258b7ba88dce5bf35ba3e94c436a5951bd917480c37da6936a8cd49c8323"} Jul 31 19:32:21.311496 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:21.311322 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:21.312408 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:21.312387 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6" Jul 31 19:32:21.979669 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:21.979627 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd"] Jul 31 19:32:21.979891 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:21.979862 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" podUID="1835f4b8-db03-4005-99df-bbc9406dd239" containerName="discovery" containerID="cri-o://2376b02850af0ce4b03ce33f36e6709847736effaa65efa310b08e27d15fa703" gracePeriod=30 Jul 31 19:32:22.318923 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.318834 2565 generic.go:356] "Generic (PLEG): container finished" podID="1835f4b8-db03-4005-99df-bbc9406dd239" containerID="2376b02850af0ce4b03ce33f36e6709847736effaa65efa310b08e27d15fa703" exitCode=0 Jul 31 19:32:22.318923 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.318872 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" event={"ID":"1835f4b8-db03-4005-99df-bbc9406dd239","Type":"ContainerDied","Data":"2376b02850af0ce4b03ce33f36e6709847736effaa65efa310b08e27d15fa703"} Jul 31 19:32:22.719482 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.719458 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:22.747882 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.747852 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1835f4b8-db03-4005-99df-bbc9406dd239-local-certs\") pod \"1835f4b8-db03-4005-99df-bbc9406dd239\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " Jul 31 19:32:22.748075 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.747918 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-kubeconfig\") pod \"1835f4b8-db03-4005-99df-bbc9406dd239\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " Jul 31 19:32:22.748075 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.747978 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-ca-configmap\") pod \"1835f4b8-db03-4005-99df-bbc9406dd239\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " Jul 31 19:32:22.748075 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.748017 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2w2zt\" (UniqueName: \"kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-kube-api-access-2w2zt\") pod \"1835f4b8-db03-4005-99df-bbc9406dd239\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " Jul 31 19:32:22.748075 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.748054 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-dns-cert\") pod \"1835f4b8-db03-4005-99df-bbc9406dd239\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " Jul 31 19:32:22.748274 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.748103 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-istio-token\") pod \"1835f4b8-db03-4005-99df-bbc9406dd239\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " Jul 31 19:32:22.748274 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.748130 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-cacerts\") pod \"1835f4b8-db03-4005-99df-bbc9406dd239\" (UID: \"1835f4b8-db03-4005-99df-bbc9406dd239\") " Jul 31 19:32:22.748482 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.748432 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "1835f4b8-db03-4005-99df-bbc9406dd239" (UID: "1835f4b8-db03-4005-99df-bbc9406dd239"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 19:32:22.751139 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.751099 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "1835f4b8-db03-4005-99df-bbc9406dd239" (UID: "1835f4b8-db03-4005-99df-bbc9406dd239"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 19:32:22.751389 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.751232 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-istio-token" (OuterVolumeSpecName: "istio-token") pod "1835f4b8-db03-4005-99df-bbc9406dd239" (UID: "1835f4b8-db03-4005-99df-bbc9406dd239"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 19:32:22.751726 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.751688 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-kube-api-access-2w2zt" (OuterVolumeSpecName: "kube-api-access-2w2zt") pod "1835f4b8-db03-4005-99df-bbc9406dd239" (UID: "1835f4b8-db03-4005-99df-bbc9406dd239"). InnerVolumeSpecName "kube-api-access-2w2zt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 19:32:22.751726 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.751702 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-cacerts" (OuterVolumeSpecName: "cacerts") pod "1835f4b8-db03-4005-99df-bbc9406dd239" (UID: "1835f4b8-db03-4005-99df-bbc9406dd239"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 19:32:22.752151 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.752130 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1835f4b8-db03-4005-99df-bbc9406dd239-local-certs" (OuterVolumeSpecName: "local-certs") pod "1835f4b8-db03-4005-99df-bbc9406dd239" (UID: "1835f4b8-db03-4005-99df-bbc9406dd239"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 19:32:22.752200 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.752136 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "1835f4b8-db03-4005-99df-bbc9406dd239" (UID: "1835f4b8-db03-4005-99df-bbc9406dd239"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 19:32:22.849166 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.849125 2565 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1835f4b8-db03-4005-99df-bbc9406dd239-local-certs\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:32:22.849166 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.849156 2565 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-kubeconfig\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:32:22.849166 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.849171 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-ca-configmap\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:32:22.849457 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.849183 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w2zt\" (UniqueName: \"kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-kube-api-access-2w2zt\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:32:22.849457 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.849194 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-istio-csr-dns-cert\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:32:22.849457 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.849205 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1835f4b8-db03-4005-99df-bbc9406dd239-istio-token\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:32:22.849457 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:22.849220 2565 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1835f4b8-db03-4005-99df-bbc9406dd239-cacerts\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:32:23.327358 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:23.327304 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" Jul 31 19:32:23.327358 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:23.327305 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" event={"ID":"1835f4b8-db03-4005-99df-bbc9406dd239","Type":"ContainerDied","Data":"445251782e6f6bc63d675786af1562e00850e030b9fa93023ee5d7b1fdda55ca"} Jul 31 19:32:23.327884 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:23.327419 2565 scope.go:117] "RemoveContainer" containerID="2376b02850af0ce4b03ce33f36e6709847736effaa65efa310b08e27d15fa703" Jul 31 19:32:23.331082 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:23.330973 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" event={"ID":"1b2f107d-07cd-4d05-a7f3-c40d3c30162b","Type":"ContainerStarted","Data":"ed28466170f9b1f52bf23eea25577a7c3f734b0486edecddf003c6ff6ada7be9"} Jul 31 19:32:23.331834 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:23.331266 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" Jul 31 19:32:23.347585 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:23.347547 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd"] Jul 31 19:32:23.351658 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:23.351587 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd"] Jul 31 19:32:23.367453 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:23.367396 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" podStartSLOduration=1.596532244 podStartE2EDuration="4.367373958s" totalImagesPullingTime="2.770841714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:32:23.366122676 +0000 UTC m=+538.637470056" watchObservedRunningTime="2026-07-31 19:32:23.367373958 +0000 UTC m=+538.638721314" Jul 31 19:32:24.337592 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:24.337560 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" event={"ID":"c3ee0c60-5559-42b2-93a6-069acf139e49","Type":"ContainerStarted","Data":"a04c16f64029add2bb5a55e9dac6df2006f1367f513b4a2ab21efded2568f8e3"} Jul 31 19:32:24.338018 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:24.337685 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" Jul 31 19:32:24.371586 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:24.371530 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" podStartSLOduration=2.272451845 podStartE2EDuration="5.371510491s" totalImagesPullingTime="3.099058646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:32:24.369112361 +0000 UTC m=+539.640459716" watchObservedRunningTime="2026-07-31 19:32:24.371510491 +0000 UTC m=+539.642857845" Jul 31 19:32:25.265329 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:25.265293 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1835f4b8-db03-4005-99df-bbc9406dd239" path="/var/lib/kubelet/pods/1835f4b8-db03-4005-99df-bbc9406dd239/volumes" Jul 31 19:32:27.276537 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:27.276495 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-49kjd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.25:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 31 19:32:27.276954 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:27.276579 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49kjd" podUID="1835f4b8-db03-4005-99df-bbc9406dd239" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.25:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 31 19:32:27.282661 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:27.282629 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kqxl5" Jul 31 19:32:27.284561 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:27.284538 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-krdgh" Jul 31 19:32:54.342614 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:54.342576 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nshht" Jul 31 19:32:55.344261 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:32:55.344228 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jcw7n" Jul 31 19:33:23.392258 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.392223 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-6lmnx"] Jul 31 19:33:23.392635 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.392594 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1835f4b8-db03-4005-99df-bbc9406dd239" containerName="discovery" Jul 31 19:33:23.392635 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.392606 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1835f4b8-db03-4005-99df-bbc9406dd239" containerName="discovery" Jul 31 19:33:23.392752 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.392671 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1835f4b8-db03-4005-99df-bbc9406dd239" containerName="discovery" Jul 31 19:33:23.395562 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.395544 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-6lmnx" Jul 31 19:33:23.397673 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.397649 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-cnlc7\"" Jul 31 19:33:23.397790 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.397686 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 31 19:33:23.397790 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.397738 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 31 19:33:23.404359 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.404320 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-6lmnx"] Jul 31 19:33:23.468209 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.468170 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7r8b\" (UniqueName: \"kubernetes.io/projected/d76b395e-3f70-44c1-b8b0-8077db201543-kube-api-access-g7r8b\") pod \"authorino-operator-68d88948d8-6lmnx\" (UID: \"d76b395e-3f70-44c1-b8b0-8077db201543\") " pod="kuadrant-system/authorino-operator-68d88948d8-6lmnx" Jul 31 19:33:23.569278 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.569244 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7r8b\" (UniqueName: \"kubernetes.io/projected/d76b395e-3f70-44c1-b8b0-8077db201543-kube-api-access-g7r8b\") pod \"authorino-operator-68d88948d8-6lmnx\" (UID: \"d76b395e-3f70-44c1-b8b0-8077db201543\") " pod="kuadrant-system/authorino-operator-68d88948d8-6lmnx" Jul 31 19:33:23.585668 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.585630 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7r8b\" (UniqueName: \"kubernetes.io/projected/d76b395e-3f70-44c1-b8b0-8077db201543-kube-api-access-g7r8b\") pod \"authorino-operator-68d88948d8-6lmnx\" (UID: \"d76b395e-3f70-44c1-b8b0-8077db201543\") " pod="kuadrant-system/authorino-operator-68d88948d8-6lmnx" Jul 31 19:33:23.707329 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.707238 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-6lmnx" Jul 31 19:33:23.842109 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:23.842084 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-6lmnx"] Jul 31 19:33:23.844647 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:33:23.844624 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76b395e_3f70_44c1_b8b0_8077db201543.slice/crio-198f6192545ece6f7a1fd6b1e28a919b3bc067ec1e3bfd816d0948dccee5970f WatchSource:0}: Error finding container 198f6192545ece6f7a1fd6b1e28a919b3bc067ec1e3bfd816d0948dccee5970f: Status 404 returned error can't find the container with id 198f6192545ece6f7a1fd6b1e28a919b3bc067ec1e3bfd816d0948dccee5970f Jul 31 19:33:24.618638 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:24.618582 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-6lmnx" event={"ID":"d76b395e-3f70-44c1-b8b0-8077db201543","Type":"ContainerStarted","Data":"198f6192545ece6f7a1fd6b1e28a919b3bc067ec1e3bfd816d0948dccee5970f"} Jul 31 19:33:25.688039 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:25.687610 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d"] Jul 31 19:33:25.698925 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:25.696202 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d" Jul 31 19:33:25.699181 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:25.699099 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d"] Jul 31 19:33:25.699639 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:25.699438 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 31 19:33:25.700855 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:25.700651 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-mvkh6\"" Jul 31 19:33:25.788814 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:25.788777 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9789\" (UniqueName: \"kubernetes.io/projected/fd0713f0-ddbe-4ebd-8aeb-34d636cee19f-kube-api-access-m9789\") pod \"dns-operator-controller-manager-5985564776-k2b7d\" (UID: \"fd0713f0-ddbe-4ebd-8aeb-34d636cee19f\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d" Jul 31 19:33:25.890254 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:25.890211 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9789\" (UniqueName: \"kubernetes.io/projected/fd0713f0-ddbe-4ebd-8aeb-34d636cee19f-kube-api-access-m9789\") pod \"dns-operator-controller-manager-5985564776-k2b7d\" (UID: \"fd0713f0-ddbe-4ebd-8aeb-34d636cee19f\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d" Jul 31 19:33:25.903624 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:25.903590 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9789\" (UniqueName: \"kubernetes.io/projected/fd0713f0-ddbe-4ebd-8aeb-34d636cee19f-kube-api-access-m9789\") pod \"dns-operator-controller-manager-5985564776-k2b7d\" (UID: \"fd0713f0-ddbe-4ebd-8aeb-34d636cee19f\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d" Jul 31 19:33:26.012267 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:26.012179 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d" Jul 31 19:33:26.410484 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:26.410462 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d"] Jul 31 19:33:26.412816 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:33:26.412792 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0713f0_ddbe_4ebd_8aeb_34d636cee19f.slice/crio-a287c1fde73eaeeddd6ed7ad1655d8ac4229d6a6224dbe214d319ef1565008e1 WatchSource:0}: Error finding container a287c1fde73eaeeddd6ed7ad1655d8ac4229d6a6224dbe214d319ef1565008e1: Status 404 returned error can't find the container with id a287c1fde73eaeeddd6ed7ad1655d8ac4229d6a6224dbe214d319ef1565008e1 Jul 31 19:33:26.635405 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:26.635359 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-6lmnx" event={"ID":"d76b395e-3f70-44c1-b8b0-8077db201543","Type":"ContainerStarted","Data":"6381175ae1412a50e20987588f265a50c7d9e2b44f8f937f9970f331d03ad25b"} Jul 31 19:33:26.635617 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:26.635447 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-6lmnx" Jul 31 19:33:26.637113 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:26.637075 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d" event={"ID":"fd0713f0-ddbe-4ebd-8aeb-34d636cee19f","Type":"ContainerStarted","Data":"a287c1fde73eaeeddd6ed7ad1655d8ac4229d6a6224dbe214d319ef1565008e1"} Jul 31 19:33:26.652386 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:26.652321 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-6lmnx" podStartSLOduration=1.162680021 podStartE2EDuration="3.652304698s" totalImagesPullingTime="2.489624677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:33:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:33:26.650976515 +0000 UTC m=+601.922323847" watchObservedRunningTime="2026-07-31 19:33:26.652304698 +0000 UTC m=+601.923652054" Jul 31 19:33:28.119902 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.119872 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n"] Jul 31 19:33:28.130423 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.130395 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:28.132695 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.132673 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-gmj9n\"" Jul 31 19:33:28.137041 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.137016 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n"] Jul 31 19:33:28.214061 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.214027 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krj45\" (UniqueName: \"kubernetes.io/projected/601be98e-313a-4602-bf54-cffc358a1444-kube-api-access-krj45\") pod \"limitador-operator-controller-manager-6c549b6d85-nl45n\" (UID: \"601be98e-313a-4602-bf54-cffc358a1444\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:28.314627 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.314592 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krj45\" (UniqueName: \"kubernetes.io/projected/601be98e-313a-4602-bf54-cffc358a1444-kube-api-access-krj45\") pod \"limitador-operator-controller-manager-6c549b6d85-nl45n\" (UID: \"601be98e-313a-4602-bf54-cffc358a1444\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:28.326905 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.326876 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krj45\" (UniqueName: \"kubernetes.io/projected/601be98e-313a-4602-bf54-cffc358a1444-kube-api-access-krj45\") pod \"limitador-operator-controller-manager-6c549b6d85-nl45n\" (UID: \"601be98e-313a-4602-bf54-cffc358a1444\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:28.455795 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.455700 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:28.607877 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.606917 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n"] Jul 31 19:33:28.618582 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:33:28.618547 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod601be98e_313a_4602_bf54_cffc358a1444.slice/crio-fd8fd0f89efd31693df16541095a237a2cc11c0ff29ea479698b0fe640264b69 WatchSource:0}: Error finding container fd8fd0f89efd31693df16541095a237a2cc11c0ff29ea479698b0fe640264b69: Status 404 returned error can't find the container with id fd8fd0f89efd31693df16541095a237a2cc11c0ff29ea479698b0fe640264b69 Jul 31 19:33:28.650766 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.650724 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" event={"ID":"601be98e-313a-4602-bf54-cffc358a1444","Type":"ContainerStarted","Data":"fd8fd0f89efd31693df16541095a237a2cc11c0ff29ea479698b0fe640264b69"} Jul 31 19:33:28.652887 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.652853 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d" event={"ID":"fd0713f0-ddbe-4ebd-8aeb-34d636cee19f","Type":"ContainerStarted","Data":"02cfbc8687a498a8a2e024db819f2b417d5bf5008fbdee6a71daa7643d5a7f1b"} Jul 31 19:33:28.653048 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.652945 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d" Jul 31 19:33:28.669752 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:28.669705 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d" podStartSLOduration=1.9795089049999999 podStartE2EDuration="3.66968581s" totalImagesPullingTime="1.690176905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:33:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:33:28.668653939 +0000 UTC m=+603.940001296" watchObservedRunningTime="2026-07-31 19:33:28.66968581 +0000 UTC m=+603.941033164" Jul 31 19:33:30.664808 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:30.664770 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" event={"ID":"601be98e-313a-4602-bf54-cffc358a1444","Type":"ContainerStarted","Data":"0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22"} Jul 31 19:33:30.665187 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:30.664840 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:30.682821 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:30.682766 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" podStartSLOduration=1.184371502 podStartE2EDuration="2.682750423s" totalImagesPullingTime="1.498378921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:33:30.682046308 +0000 UTC m=+605.953393663" watchObservedRunningTime="2026-07-31 19:33:30.682750423 +0000 UTC m=+605.954097778" Jul 31 19:33:37.644536 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:37.644504 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-6lmnx" Jul 31 19:33:39.340882 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.340849 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7"] Jul 31 19:33:39.344724 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.344702 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:39.346674 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.346652 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 31 19:33:39.346793 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.346697 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 31 19:33:39.346793 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.346701 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xrw9x\"" Jul 31 19:33:39.352893 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.352870 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7"] Jul 31 19:33:39.420264 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.420230 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8f4d\" (UniqueName: \"kubernetes.io/projected/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-kube-api-access-h8f4d\") pod \"kuadrant-console-plugin-598f86468b-td4z7\" (UID: \"bb36e754-d9e8-4528-8e48-e5bf33d8ceee\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:39.420454 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.420310 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-td4z7\" (UID: \"bb36e754-d9e8-4528-8e48-e5bf33d8ceee\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:39.420454 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.420372 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-td4z7\" (UID: \"bb36e754-d9e8-4528-8e48-e5bf33d8ceee\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:39.521716 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.521679 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-td4z7\" (UID: \"bb36e754-d9e8-4528-8e48-e5bf33d8ceee\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:39.521716 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.521729 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8f4d\" (UniqueName: \"kubernetes.io/projected/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-kube-api-access-h8f4d\") pod \"kuadrant-console-plugin-598f86468b-td4z7\" (UID: \"bb36e754-d9e8-4528-8e48-e5bf33d8ceee\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:39.521943 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:39.521829 2565 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 31 19:33:39.521943 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:39.521900 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-plugin-serving-cert podName:bb36e754-d9e8-4528-8e48-e5bf33d8ceee nodeName:}" failed. No retries permitted until 2026-07-31 19:33:40.021884207 +0000 UTC m=+615.293231543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-plugin-serving-cert") pod "kuadrant-console-plugin-598f86468b-td4z7" (UID: "bb36e754-d9e8-4528-8e48-e5bf33d8ceee") : secret "plugin-serving-cert" not found Jul 31 19:33:39.521943 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.521918 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-td4z7\" (UID: \"bb36e754-d9e8-4528-8e48-e5bf33d8ceee\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:39.522539 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.522523 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-td4z7\" (UID: \"bb36e754-d9e8-4528-8e48-e5bf33d8ceee\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:39.531941 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.531923 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8f4d\" (UniqueName: \"kubernetes.io/projected/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-kube-api-access-h8f4d\") pod \"kuadrant-console-plugin-598f86468b-td4z7\" (UID: \"bb36e754-d9e8-4528-8e48-e5bf33d8ceee\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:39.661449 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:39.661336 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-k2b7d" Jul 31 19:33:40.026738 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:40.026653 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-td4z7\" (UID: \"bb36e754-d9e8-4528-8e48-e5bf33d8ceee\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:40.029279 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:40.029250 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb36e754-d9e8-4528-8e48-e5bf33d8ceee-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-td4z7\" (UID: \"bb36e754-d9e8-4528-8e48-e5bf33d8ceee\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:40.254545 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:40.254515 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" Jul 31 19:33:40.397176 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:40.397145 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7"] Jul 31 19:33:40.399846 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:33:40.399808 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb36e754_d9e8_4528_8e48_e5bf33d8ceee.slice/crio-f919f5352ef425c66d513c464eb2de29da132b4cbefed17d495a061db581a73e WatchSource:0}: Error finding container f919f5352ef425c66d513c464eb2de29da132b4cbefed17d495a061db581a73e: Status 404 returned error can't find the container with id f919f5352ef425c66d513c464eb2de29da132b4cbefed17d495a061db581a73e Jul 31 19:33:40.723412 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:40.723375 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" event={"ID":"bb36e754-d9e8-4528-8e48-e5bf33d8ceee","Type":"ContainerStarted","Data":"f919f5352ef425c66d513c464eb2de29da132b4cbefed17d495a061db581a73e"} Jul 31 19:33:41.673216 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:41.673183 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:45.754486 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:45.754444 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" event={"ID":"bb36e754-d9e8-4528-8e48-e5bf33d8ceee","Type":"ContainerStarted","Data":"bad82a864187011d74671b6aa56719f3c0a1e9845ed0540cb831dab0cb1163dd"} Jul 31 19:33:45.769971 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:45.769922 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-td4z7" podStartSLOduration=2.187236805 podStartE2EDuration="6.769904489s" totalImagesPullingTime="4.582667684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:33:45.767781625 +0000 UTC m=+621.039128980" watchObservedRunningTime="2026-07-31 19:33:45.769904489 +0000 UTC m=+621.041251844" Jul 31 19:33:48.792753 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:48.792717 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz"] Jul 31 19:33:48.796437 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:48.796404 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:48.798754 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:48.798735 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kf8bl\"" Jul 31 19:33:48.808739 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:48.808711 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz"] Jul 31 19:33:48.897017 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:48.896986 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz"] Jul 31 19:33:48.897367 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:48.897317 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-jn94g], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" podUID="157aeb61-670e-4c30-860d-84799e56c6a2" Jul 31 19:33:48.910401 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:48.910364 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn94g\" (UniqueName: \"kubernetes.io/projected/157aeb61-670e-4c30-860d-84799e56c6a2-kube-api-access-jn94g\") pod \"kuadrant-operator-controller-manager-96db98c47-hn7gz\" (UID: \"157aeb61-670e-4c30-860d-84799e56c6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:48.910520 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:48.910456 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/157aeb61-670e-4c30-860d-84799e56c6a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-hn7gz\" (UID: \"157aeb61-670e-4c30-860d-84799e56c6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:49.011091 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.011047 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/157aeb61-670e-4c30-860d-84799e56c6a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-hn7gz\" (UID: \"157aeb61-670e-4c30-860d-84799e56c6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:49.011303 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.011123 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn94g\" (UniqueName: \"kubernetes.io/projected/157aeb61-670e-4c30-860d-84799e56c6a2-kube-api-access-jn94g\") pod \"kuadrant-operator-controller-manager-96db98c47-hn7gz\" (UID: \"157aeb61-670e-4c30-860d-84799e56c6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:49.011540 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.011517 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/157aeb61-670e-4c30-860d-84799e56c6a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-hn7gz\" (UID: \"157aeb61-670e-4c30-860d-84799e56c6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:49.019605 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.019584 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn94g\" (UniqueName: \"kubernetes.io/projected/157aeb61-670e-4c30-860d-84799e56c6a2-kube-api-access-jn94g\") pod \"kuadrant-operator-controller-manager-96db98c47-hn7gz\" (UID: \"157aeb61-670e-4c30-860d-84799e56c6a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:49.428434 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.428403 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz"] Jul 31 19:33:49.439530 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.439502 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz"] Jul 31 19:33:49.450968 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.450942 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n"] Jul 31 19:33:49.451175 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.451152 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" podUID="601be98e-313a-4602-bf54-cffc358a1444" containerName="manager" containerID="cri-o://0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22" gracePeriod=2 Jul 31 19:33:49.461673 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.461646 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n"] Jul 31 19:33:49.463950 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:49.463918 2565 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"601be98e-313a-4602-bf54-cffc358a1444\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-6c549b6d85-nl45n\": pods \"limitador-operator-controller-manager-6c549b6d85-nl45n\" not found" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:49.473595 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.473571 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp"] Jul 31 19:33:49.473984 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.473972 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="601be98e-313a-4602-bf54-cffc358a1444" containerName="manager" Jul 31 19:33:49.474032 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.473987 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="601be98e-313a-4602-bf54-cffc358a1444" containerName="manager" Jul 31 19:33:49.474066 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.474050 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="601be98e-313a-4602-bf54-cffc358a1444" containerName="manager" Jul 31 19:33:49.476911 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.476895 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp" Jul 31 19:33:49.478784 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:49.478753 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-nl45n\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="601be98e-313a-4602-bf54-cffc358a1444" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:49.487029 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.487005 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp"] Jul 31 19:33:49.617860 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.617827 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grjlx\" (UniqueName: \"kubernetes.io/projected/46f004da-30a5-4764-a582-8a6ddaa7bec6-kube-api-access-grjlx\") pod \"limitador-operator-controller-manager-6c549b6d85-lvlmp\" (UID: \"46f004da-30a5-4764-a582-8a6ddaa7bec6\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp" Jul 31 19:33:49.699074 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.699041 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:49.701028 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:49.700995 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-nl45n\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="601be98e-313a-4602-bf54-cffc358a1444" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:49.718541 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.718516 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grjlx\" (UniqueName: \"kubernetes.io/projected/46f004da-30a5-4764-a582-8a6ddaa7bec6-kube-api-access-grjlx\") pod \"limitador-operator-controller-manager-6c549b6d85-lvlmp\" (UID: \"46f004da-30a5-4764-a582-8a6ddaa7bec6\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp" Jul 31 19:33:49.727260 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.727233 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grjlx\" (UniqueName: \"kubernetes.io/projected/46f004da-30a5-4764-a582-8a6ddaa7bec6-kube-api-access-grjlx\") pod \"limitador-operator-controller-manager-6c549b6d85-lvlmp\" (UID: \"46f004da-30a5-4764-a582-8a6ddaa7bec6\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp" Jul 31 19:33:49.775829 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.775796 2565 generic.go:356] "Generic (PLEG): container finished" podID="601be98e-313a-4602-bf54-cffc358a1444" containerID="0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22" exitCode=0 Jul 31 19:33:49.776025 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.775843 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:49.776025 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.775855 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:49.776025 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.775897 2565 scope.go:117] "RemoveContainer" containerID="0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22" Jul 31 19:33:49.778113 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:49.778084 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-nl45n\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="601be98e-313a-4602-bf54-cffc358a1444" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:49.781964 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:49.781940 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-hn7gz\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="157aeb61-670e-4c30-860d-84799e56c6a2" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:49.783531 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.783507 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:49.783634 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:49.783528 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-nl45n\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="601be98e-313a-4602-bf54-cffc358a1444" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:49.785148 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:49.785124 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-hn7gz\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="157aeb61-670e-4c30-860d-84799e56c6a2" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:49.785417 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.785404 2565 scope.go:117] "RemoveContainer" containerID="0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22" Jul 31 19:33:49.785660 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:49.785644 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22\": container with ID starting with 0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22 not found: ID does not exist" containerID="0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22" Jul 31 19:33:49.785723 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.785667 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22"} err="failed to get container status \"0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22\": rpc error: code = NotFound desc = could not find container \"0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22\": container with ID starting with 0497fa93fcb0a86b1b2cd7511533a73bdebd885d2b209b624dbbe63e90543b22 not found: ID does not exist" Jul 31 19:33:49.786751 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:49.786732 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-nl45n\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="601be98e-313a-4602-bf54-cffc358a1444" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:49.819013 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.818984 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krj45\" (UniqueName: \"kubernetes.io/projected/601be98e-313a-4602-bf54-cffc358a1444-kube-api-access-krj45\") pod \"601be98e-313a-4602-bf54-cffc358a1444\" (UID: \"601be98e-313a-4602-bf54-cffc358a1444\") " Jul 31 19:33:49.821237 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.821217 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/601be98e-313a-4602-bf54-cffc358a1444-kube-api-access-krj45" (OuterVolumeSpecName: "kube-api-access-krj45") pod "601be98e-313a-4602-bf54-cffc358a1444" (UID: "601be98e-313a-4602-bf54-cffc358a1444"). InnerVolumeSpecName "kube-api-access-krj45". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 19:33:49.834415 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.834392 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp" Jul 31 19:33:49.919668 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.919644 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn94g\" (UniqueName: \"kubernetes.io/projected/157aeb61-670e-4c30-860d-84799e56c6a2-kube-api-access-jn94g\") pod \"157aeb61-670e-4c30-860d-84799e56c6a2\" (UID: \"157aeb61-670e-4c30-860d-84799e56c6a2\") " Jul 31 19:33:49.919764 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.919722 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/157aeb61-670e-4c30-860d-84799e56c6a2-extensions-socket-volume\") pod \"157aeb61-670e-4c30-860d-84799e56c6a2\" (UID: \"157aeb61-670e-4c30-860d-84799e56c6a2\") " Jul 31 19:33:49.919962 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.919930 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krj45\" (UniqueName: \"kubernetes.io/projected/601be98e-313a-4602-bf54-cffc358a1444-kube-api-access-krj45\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:33:49.920070 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.920021 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/157aeb61-670e-4c30-860d-84799e56c6a2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "157aeb61-670e-4c30-860d-84799e56c6a2" (UID: "157aeb61-670e-4c30-860d-84799e56c6a2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 19:33:49.922429 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.922404 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/157aeb61-670e-4c30-860d-84799e56c6a2-kube-api-access-jn94g" (OuterVolumeSpecName: "kube-api-access-jn94g") pod "157aeb61-670e-4c30-860d-84799e56c6a2" (UID: "157aeb61-670e-4c30-860d-84799e56c6a2"). InnerVolumeSpecName "kube-api-access-jn94g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 19:33:49.974131 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:49.974107 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp"] Jul 31 19:33:49.976206 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:33:49.976179 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46f004da_30a5_4764_a582_8a6ddaa7bec6.slice/crio-2b718fea6b3ab5cdbb00cb4fc487c374766d32b2c6b70f1248d5d6253b03b478 WatchSource:0}: Error finding container 2b718fea6b3ab5cdbb00cb4fc487c374766d32b2c6b70f1248d5d6253b03b478: Status 404 returned error can't find the container with id 2b718fea6b3ab5cdbb00cb4fc487c374766d32b2c6b70f1248d5d6253b03b478 Jul 31 19:33:50.020413 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:50.020388 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn94g\" (UniqueName: \"kubernetes.io/projected/157aeb61-670e-4c30-860d-84799e56c6a2-kube-api-access-jn94g\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:33:50.020413 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:50.020411 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/157aeb61-670e-4c30-860d-84799e56c6a2-extensions-socket-volume\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:33:50.086152 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:50.086121 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-hn7gz\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="157aeb61-670e-4c30-860d-84799e56c6a2" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:50.087754 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:50.087730 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-nl45n\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="601be98e-313a-4602-bf54-cffc358a1444" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:50.783237 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:50.783210 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:50.783460 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:50.783253 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp" event={"ID":"46f004da-30a5-4764-a582-8a6ddaa7bec6","Type":"ContainerStarted","Data":"d44a79a4127f8b92d90db89f2fd6c3361d311a8ff6d1dcfa86861c700c001a8a"} Jul 31 19:33:50.783460 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:50.783303 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp" event={"ID":"46f004da-30a5-4764-a582-8a6ddaa7bec6","Type":"ContainerStarted","Data":"2b718fea6b3ab5cdbb00cb4fc487c374766d32b2c6b70f1248d5d6253b03b478"} Jul 31 19:33:50.783460 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:50.783458 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp" Jul 31 19:33:50.785410 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:50.785377 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-hn7gz\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="157aeb61-670e-4c30-860d-84799e56c6a2" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:50.787066 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:50.787047 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-nl45n\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="601be98e-313a-4602-bf54-cffc358a1444" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:50.807206 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:50.807158 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp" podStartSLOduration=1.8071386729999999 podStartE2EDuration="1.807138673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:33:50.806969161 +0000 UTC m=+626.078316541" watchObservedRunningTime="2026-07-31 19:33:50.807138673 +0000 UTC m=+626.078486028" Jul 31 19:33:50.808755 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:50.808726 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-hn7gz\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="157aeb61-670e-4c30-860d-84799e56c6a2" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-hn7gz" Jul 31 19:33:50.810281 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:33:50.810254 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-nl45n\" is forbidden: User \"system:node:ip-10-0-136-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-28.ec2.internal' and this object" podUID="601be98e-313a-4602-bf54-cffc358a1444" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nl45n" Jul 31 19:33:51.263915 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:51.263881 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="157aeb61-670e-4c30-860d-84799e56c6a2" path="/var/lib/kubelet/pods/157aeb61-670e-4c30-860d-84799e56c6a2/volumes" Jul 31 19:33:51.264294 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:33:51.264098 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="601be98e-313a-4602-bf54-cffc358a1444" path="/var/lib/kubelet/pods/601be98e-313a-4602-bf54-cffc358a1444/volumes" Jul 31 19:34:01.790682 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:01.790653 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lvlmp" Jul 31 19:34:21.671559 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.671520 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm"] Jul 31 19:34:21.677409 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.677384 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.679648 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.679626 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bw8lh\"" Jul 31 19:34:21.686017 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.685993 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm"] Jul 31 19:34:21.799328 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.799283 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.799328 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.799357 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.799574 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.799385 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1b041eb-d946-4c32-bb75-490e0f781091-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.799574 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.799419 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.799574 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.799435 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.799574 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.799451 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcjnk\" (UniqueName: \"kubernetes.io/projected/d1b041eb-d946-4c32-bb75-490e0f781091-kube-api-access-zcjnk\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.799574 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.799471 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1b041eb-d946-4c32-bb75-490e0f781091-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.799574 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.799547 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.799754 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.799602 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1b041eb-d946-4c32-bb75-490e0f781091-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.900538 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.900497 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.900744 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.900558 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.900744 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.900579 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1b041eb-d946-4c32-bb75-490e0f781091-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.900744 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.900619 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.900744 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.900641 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.900744 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.900664 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcjnk\" (UniqueName: \"kubernetes.io/projected/d1b041eb-d946-4c32-bb75-490e0f781091-kube-api-access-zcjnk\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.900744 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.900688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1b041eb-d946-4c32-bb75-490e0f781091-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.900744 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.900728 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.901160 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.900759 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1b041eb-d946-4c32-bb75-490e0f781091-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.901160 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.900966 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.901160 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.901077 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.901160 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.901121 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.901400 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.901193 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.901493 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.901475 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1b041eb-d946-4c32-bb75-490e0f781091-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.903167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.903149 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1b041eb-d946-4c32-bb75-490e0f781091-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.903394 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.903376 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1b041eb-d946-4c32-bb75-490e0f781091-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.907990 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.907961 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1b041eb-d946-4c32-bb75-490e0f781091-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.908237 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.908217 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcjnk\" (UniqueName: \"kubernetes.io/projected/d1b041eb-d946-4c32-bb75-490e0f781091-kube-api-access-zcjnk\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xnzmm\" (UID: \"d1b041eb-d946-4c32-bb75-490e0f781091\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:21.989201 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:21.989107 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:22.132185 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:22.132159 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm"] Jul 31 19:34:22.134294 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:34:22.134266 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1b041eb_d946_4c32_bb75_490e0f781091.slice/crio-9723b08e8da1298374fa804ee799a7a2fb4ef39f2a84927c22255db6e48654fa WatchSource:0}: Error finding container 9723b08e8da1298374fa804ee799a7a2fb4ef39f2a84927c22255db6e48654fa: Status 404 returned error can't find the container with id 9723b08e8da1298374fa804ee799a7a2fb4ef39f2a84927c22255db6e48654fa Jul 31 19:34:22.136879 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:22.136845 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 31 19:34:22.136957 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:22.136923 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 31 19:34:22.136957 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:22.136951 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 31 19:34:22.947426 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:22.947394 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" event={"ID":"d1b041eb-d946-4c32-bb75-490e0f781091","Type":"ContainerStarted","Data":"b85d0ce1157f4c573b9b06d40f518d1b3bd3397cefde1cf26694a0e152d77591"} Jul 31 19:34:22.947865 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:22.947451 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" event={"ID":"d1b041eb-d946-4c32-bb75-490e0f781091","Type":"ContainerStarted","Data":"9723b08e8da1298374fa804ee799a7a2fb4ef39f2a84927c22255db6e48654fa"} Jul 31 19:34:22.967556 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:22.967504 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" podStartSLOduration=1.9674877880000001 podStartE2EDuration="1.967487788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:34:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:34:22.965219372 +0000 UTC m=+658.236566726" watchObservedRunningTime="2026-07-31 19:34:22.967487788 +0000 UTC m=+658.238835143" Jul 31 19:34:22.989265 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:22.989224 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:23.994700 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:23.994672 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:24.958852 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:24.958826 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:24.959951 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:24.959932 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:34:34.169875 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.169837 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-d7qhd"] Jul 31 19:34:34.174629 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.174607 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:34.176565 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.176543 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 31 19:34:34.180497 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.180473 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-d7qhd"] Jul 31 19:34:34.275470 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.275429 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-d7qhd"] Jul 31 19:34:34.323287 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.323245 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-config-file\") pod \"limitador-limitador-859c4d96d6-d7qhd\" (UID: \"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:34.323503 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.323396 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq9zt\" (UniqueName: \"kubernetes.io/projected/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-kube-api-access-cq9zt\") pod \"limitador-limitador-859c4d96d6-d7qhd\" (UID: \"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:34.424721 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.424626 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq9zt\" (UniqueName: \"kubernetes.io/projected/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-kube-api-access-cq9zt\") pod \"limitador-limitador-859c4d96d6-d7qhd\" (UID: \"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:34.424876 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.424721 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-config-file\") pod \"limitador-limitador-859c4d96d6-d7qhd\" (UID: \"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:34.425367 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.425331 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-config-file\") pod \"limitador-limitador-859c4d96d6-d7qhd\" (UID: \"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:34.432456 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.432432 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq9zt\" (UniqueName: \"kubernetes.io/projected/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-kube-api-access-cq9zt\") pod \"limitador-limitador-859c4d96d6-d7qhd\" (UID: \"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:34.486749 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.486721 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:34.629127 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:34.629099 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-d7qhd"] Jul 31 19:34:34.630841 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:34:34.630819 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e7fc8b7_23d9_49fb_b8fe_da51a13a9788.slice/crio-70af7632572b322e8a3431246bf129f7824e10f7a94597afd2908a664d58f585 WatchSource:0}: Error finding container 70af7632572b322e8a3431246bf129f7824e10f7a94597afd2908a664d58f585: Status 404 returned error can't find the container with id 70af7632572b322e8a3431246bf129f7824e10f7a94597afd2908a664d58f585 Jul 31 19:34:35.007116 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:35.007084 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" event={"ID":"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788","Type":"ContainerStarted","Data":"70af7632572b322e8a3431246bf129f7824e10f7a94597afd2908a664d58f585"} Jul 31 19:34:37.019423 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:37.019381 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" event={"ID":"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788","Type":"ContainerStarted","Data":"73ae5bf94ee89b3abd639151b0be6d3cbb16076b9ba91db89a94ff6a8ca7d006"} Jul 31 19:34:37.019907 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:37.019487 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:37.035367 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:37.035291 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" podStartSLOduration=1.68403143 podStartE2EDuration="3.03527273s" totalImagesPullingTime="1.3512413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:34:37.033597876 +0000 UTC m=+672.304945229" watchObservedRunningTime="2026-07-31 19:34:37.03527273 +0000 UTC m=+672.306620085" Jul 31 19:34:47.588602 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:47.588562 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-d7qhd"] Jul 31 19:34:47.589020 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:47.588744 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" podUID="7e7fc8b7-23d9-49fb-b8fe-da51a13a9788" containerName="limitador" containerID="cri-o://73ae5bf94ee89b3abd639151b0be6d3cbb16076b9ba91db89a94ff6a8ca7d006" gracePeriod=30 Jul 31 19:34:47.590044 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:47.589673 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:48.075364 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:48.075305 2565 generic.go:356] "Generic (PLEG): container finished" podID="7e7fc8b7-23d9-49fb-b8fe-da51a13a9788" containerID="73ae5bf94ee89b3abd639151b0be6d3cbb16076b9ba91db89a94ff6a8ca7d006" exitCode=0 Jul 31 19:34:48.075527 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:48.075382 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" event={"ID":"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788","Type":"ContainerDied","Data":"73ae5bf94ee89b3abd639151b0be6d3cbb16076b9ba91db89a94ff6a8ca7d006"} Jul 31 19:34:48.150869 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:48.150809 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:48.246077 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:48.246047 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-config-file\") pod \"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788\" (UID: \"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788\") " Jul 31 19:34:48.246260 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:48.246126 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cq9zt\" (UniqueName: \"kubernetes.io/projected/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-kube-api-access-cq9zt\") pod \"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788\" (UID: \"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788\") " Jul 31 19:34:48.246552 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:48.246526 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-config-file" (OuterVolumeSpecName: "config-file") pod "7e7fc8b7-23d9-49fb-b8fe-da51a13a9788" (UID: "7e7fc8b7-23d9-49fb-b8fe-da51a13a9788"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 19:34:48.248419 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:48.248400 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-kube-api-access-cq9zt" (OuterVolumeSpecName: "kube-api-access-cq9zt") pod "7e7fc8b7-23d9-49fb-b8fe-da51a13a9788" (UID: "7e7fc8b7-23d9-49fb-b8fe-da51a13a9788"). InnerVolumeSpecName "kube-api-access-cq9zt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 19:34:48.347223 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:48.347187 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cq9zt\" (UniqueName: \"kubernetes.io/projected/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-kube-api-access-cq9zt\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:34:48.347223 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:48.347216 2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788-config-file\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:34:49.081982 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:49.081955 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" Jul 31 19:34:49.081982 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:49.081957 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-d7qhd" event={"ID":"7e7fc8b7-23d9-49fb-b8fe-da51a13a9788","Type":"ContainerDied","Data":"70af7632572b322e8a3431246bf129f7824e10f7a94597afd2908a664d58f585"} Jul 31 19:34:49.082516 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:49.082018 2565 scope.go:117] "RemoveContainer" containerID="73ae5bf94ee89b3abd639151b0be6d3cbb16076b9ba91db89a94ff6a8ca7d006" Jul 31 19:34:49.102881 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:49.102856 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-d7qhd"] Jul 31 19:34:49.110415 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:49.108930 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-d7qhd"] Jul 31 19:34:49.264626 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:49.264594 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e7fc8b7-23d9-49fb-b8fe-da51a13a9788" path="/var/lib/kubelet/pods/7e7fc8b7-23d9-49fb-b8fe-da51a13a9788/volumes" Jul 31 19:34:50.626214 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.626177 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs"] Jul 31 19:34:50.626860 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.626835 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e7fc8b7-23d9-49fb-b8fe-da51a13a9788" containerName="limitador" Jul 31 19:34:50.627062 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.627043 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e7fc8b7-23d9-49fb-b8fe-da51a13a9788" containerName="limitador" Jul 31 19:34:50.627221 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.627207 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e7fc8b7-23d9-49fb-b8fe-da51a13a9788" containerName="limitador" Jul 31 19:34:50.632150 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.632129 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" Jul 31 19:34:50.634502 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.634476 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-6j65w\"" Jul 31 19:34:50.634612 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.634485 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 31 19:34:50.635051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.634775 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 31 19:34:50.635051 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.634784 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 31 19:34:50.636186 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.636165 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs"] Jul 31 19:34:50.667737 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.667698 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zblrk\" (UniqueName: \"kubernetes.io/projected/fe8480f6-5a0f-4539-ad3d-9644b5b3607d-kube-api-access-zblrk\") pod \"postgres-69fb4594cf-hz7vs\" (UID: \"fe8480f6-5a0f-4539-ad3d-9644b5b3607d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" Jul 31 19:34:50.667892 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.667813 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fe8480f6-5a0f-4539-ad3d-9644b5b3607d-data\") pod \"postgres-69fb4594cf-hz7vs\" (UID: \"fe8480f6-5a0f-4539-ad3d-9644b5b3607d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" Jul 31 19:34:50.768282 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.768254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zblrk\" (UniqueName: \"kubernetes.io/projected/fe8480f6-5a0f-4539-ad3d-9644b5b3607d-kube-api-access-zblrk\") pod \"postgres-69fb4594cf-hz7vs\" (UID: \"fe8480f6-5a0f-4539-ad3d-9644b5b3607d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" Jul 31 19:34:50.768457 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.768317 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fe8480f6-5a0f-4539-ad3d-9644b5b3607d-data\") pod \"postgres-69fb4594cf-hz7vs\" (UID: \"fe8480f6-5a0f-4539-ad3d-9644b5b3607d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" Jul 31 19:34:50.768639 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.768624 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fe8480f6-5a0f-4539-ad3d-9644b5b3607d-data\") pod \"postgres-69fb4594cf-hz7vs\" (UID: \"fe8480f6-5a0f-4539-ad3d-9644b5b3607d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" Jul 31 19:34:50.775993 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.775964 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zblrk\" (UniqueName: \"kubernetes.io/projected/fe8480f6-5a0f-4539-ad3d-9644b5b3607d-kube-api-access-zblrk\") pod \"postgres-69fb4594cf-hz7vs\" (UID: \"fe8480f6-5a0f-4539-ad3d-9644b5b3607d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" Jul 31 19:34:50.945031 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:50.944938 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" Jul 31 19:34:51.093800 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:51.093776 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs"] Jul 31 19:34:51.095632 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:34:51.095608 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe8480f6_5a0f_4539_ad3d_9644b5b3607d.slice/crio-995c1be53f60effafe9e24680a50943b2809818c1ce543cb84215cb6e5d881eb WatchSource:0}: Error finding container 995c1be53f60effafe9e24680a50943b2809818c1ce543cb84215cb6e5d881eb: Status 404 returned error can't find the container with id 995c1be53f60effafe9e24680a50943b2809818c1ce543cb84215cb6e5d881eb Jul 31 19:34:52.099487 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:52.099448 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" event={"ID":"fe8480f6-5a0f-4539-ad3d-9644b5b3607d","Type":"ContainerStarted","Data":"995c1be53f60effafe9e24680a50943b2809818c1ce543cb84215cb6e5d881eb"} Jul 31 19:34:57.132080 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:57.132046 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" event={"ID":"fe8480f6-5a0f-4539-ad3d-9644b5b3607d","Type":"ContainerStarted","Data":"8cfa6d06455cdb3585f26acfb033155d32791a766f234f8168499b7db31ad028"} Jul 31 19:34:57.132513 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:57.132149 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" Jul 31 19:34:57.148622 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:34:57.148571 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" podStartSLOduration=2.002167946 podStartE2EDuration="7.148555525s" totalImagesPullingTime="5.146387579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:34:57.146730198 +0000 UTC m=+692.418077571" watchObservedRunningTime="2026-07-31 19:34:57.148555525 +0000 UTC m=+692.419902879" Jul 31 19:35:03.166525 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:03.166495 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7vs" Jul 31 19:35:09.695243 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.695207 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f6db4dfbf-nlwhk"] Jul 31 19:35:09.713157 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.713131 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:09.715936 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.715899 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 31 19:35:09.716109 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.715960 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f6db4dfbf-nlwhk"] Jul 31 19:35:09.716298 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.716276 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 31 19:35:09.716808 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.716790 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pnjhn\"" Jul 31 19:35:09.845757 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.845708 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-webhook-cert\") pod \"maas-controller-6f6db4dfbf-nlwhk\" (UID: \"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc\") " pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:09.845935 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.845837 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nqpc\" (UniqueName: \"kubernetes.io/projected/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-kube-api-access-6nqpc\") pod \"maas-controller-6f6db4dfbf-nlwhk\" (UID: \"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc\") " pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:09.894534 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.894505 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bcf5f6d66-cvcwn"] Jul 31 19:35:09.896933 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.896913 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" Jul 31 19:35:09.907892 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.907867 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bcf5f6d66-cvcwn"] Jul 31 19:35:09.947146 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.947060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nqpc\" (UniqueName: \"kubernetes.io/projected/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-kube-api-access-6nqpc\") pod \"maas-controller-6f6db4dfbf-nlwhk\" (UID: \"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc\") " pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:09.947305 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.947151 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-webhook-cert\") pod \"maas-controller-6f6db4dfbf-nlwhk\" (UID: \"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc\") " pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:09.949781 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.949758 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-webhook-cert\") pod \"maas-controller-6f6db4dfbf-nlwhk\" (UID: \"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc\") " pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:09.954732 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:09.954702 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nqpc\" (UniqueName: \"kubernetes.io/projected/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-kube-api-access-6nqpc\") pod \"maas-controller-6f6db4dfbf-nlwhk\" (UID: \"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc\") " pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:10.025536 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:10.025483 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:10.048696 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:10.048663 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqsgt\" (UniqueName: \"kubernetes.io/projected/d2948fe6-b13c-4826-86d2-40858e0f8952-kube-api-access-fqsgt\") pod \"maas-controller-6bcf5f6d66-cvcwn\" (UID: \"d2948fe6-b13c-4826-86d2-40858e0f8952\") " pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" Jul 31 19:35:10.048853 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:10.048835 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2948fe6-b13c-4826-86d2-40858e0f8952-webhook-cert\") pod \"maas-controller-6bcf5f6d66-cvcwn\" (UID: \"d2948fe6-b13c-4826-86d2-40858e0f8952\") " pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" Jul 31 19:35:10.149485 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:10.149453 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2948fe6-b13c-4826-86d2-40858e0f8952-webhook-cert\") pod \"maas-controller-6bcf5f6d66-cvcwn\" (UID: \"d2948fe6-b13c-4826-86d2-40858e0f8952\") " pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" Jul 31 19:35:10.149638 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:10.149506 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqsgt\" (UniqueName: \"kubernetes.io/projected/d2948fe6-b13c-4826-86d2-40858e0f8952-kube-api-access-fqsgt\") pod \"maas-controller-6bcf5f6d66-cvcwn\" (UID: \"d2948fe6-b13c-4826-86d2-40858e0f8952\") " pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" Jul 31 19:35:10.152006 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:10.151981 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2948fe6-b13c-4826-86d2-40858e0f8952-webhook-cert\") pod \"maas-controller-6bcf5f6d66-cvcwn\" (UID: \"d2948fe6-b13c-4826-86d2-40858e0f8952\") " pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" Jul 31 19:35:10.156704 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:10.156678 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqsgt\" (UniqueName: \"kubernetes.io/projected/d2948fe6-b13c-4826-86d2-40858e0f8952-kube-api-access-fqsgt\") pod \"maas-controller-6bcf5f6d66-cvcwn\" (UID: \"d2948fe6-b13c-4826-86d2-40858e0f8952\") " pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" Jul 31 19:35:10.160700 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:10.160675 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f6db4dfbf-nlwhk"] Jul 31 19:35:10.162193 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:35:10.162163 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165f3dc3_a390_4b6f_a8b0_5a6bdb213bdc.slice/crio-1d4a2c24fe216464443c6092f77f64f7188f95342d6c30e00e5ac34e74d69f80 WatchSource:0}: Error finding container 1d4a2c24fe216464443c6092f77f64f7188f95342d6c30e00e5ac34e74d69f80: Status 404 returned error can't find the container with id 1d4a2c24fe216464443c6092f77f64f7188f95342d6c30e00e5ac34e74d69f80 Jul 31 19:35:10.200765 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:10.200691 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" event={"ID":"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc","Type":"ContainerStarted","Data":"1d4a2c24fe216464443c6092f77f64f7188f95342d6c30e00e5ac34e74d69f80"} Jul 31 19:35:10.207403 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:10.207381 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" Jul 31 19:35:10.346422 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:10.346399 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bcf5f6d66-cvcwn"] Jul 31 19:35:10.348784 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:35:10.348761 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2948fe6_b13c_4826_86d2_40858e0f8952.slice/crio-c763b916bdc9d6fa6983f7d72604d4defef68b5e361d793071e749d0c28b548e WatchSource:0}: Error finding container c763b916bdc9d6fa6983f7d72604d4defef68b5e361d793071e749d0c28b548e: Status 404 returned error can't find the container with id c763b916bdc9d6fa6983f7d72604d4defef68b5e361d793071e749d0c28b548e Jul 31 19:35:11.216938 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:11.216854 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" event={"ID":"d2948fe6-b13c-4826-86d2-40858e0f8952","Type":"ContainerStarted","Data":"c763b916bdc9d6fa6983f7d72604d4defef68b5e361d793071e749d0c28b548e"} Jul 31 19:35:13.228867 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:13.228825 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" event={"ID":"d2948fe6-b13c-4826-86d2-40858e0f8952","Type":"ContainerStarted","Data":"57e03d1fb2d1c343e31a31542e922eb28d9cbdc0ecc39fa69f1f7661d7888b3d"} Jul 31 19:35:13.229393 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:13.229018 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" Jul 31 19:35:13.230381 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:13.230358 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" event={"ID":"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc","Type":"ContainerStarted","Data":"4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26"} Jul 31 19:35:13.230512 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:13.230501 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:13.244778 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:13.244711 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" podStartSLOduration=1.889989876 podStartE2EDuration="4.244695971s" totalImagesPullingTime="2.354706095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:35:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:35:13.243948086 +0000 UTC m=+708.515295442" watchObservedRunningTime="2026-07-31 19:35:13.244695971 +0000 UTC m=+708.516043324" Jul 31 19:35:13.260803 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:13.260748 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" podStartSLOduration=1.726228326 podStartE2EDuration="4.260726192s" totalImagesPullingTime="2.534497866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:35:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:35:13.258702335 +0000 UTC m=+708.530049689" watchObservedRunningTime="2026-07-31 19:35:13.260726192 +0000 UTC m=+708.532073547" Jul 31 19:35:24.237820 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:24.237789 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bcf5f6d66-cvcwn" Jul 31 19:35:24.238202 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:24.237861 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:24.291766 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:24.291734 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f6db4dfbf-nlwhk"] Jul 31 19:35:24.291995 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:24.291954 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" podUID="165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc" containerName="manager" containerID="cri-o://4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26" gracePeriod=30 Jul 31 19:35:24.559419 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:24.559392 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:24.688095 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:24.688063 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-webhook-cert\") pod \"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc\" (UID: \"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc\") " Jul 31 19:35:24.688285 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:24.688155 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nqpc\" (UniqueName: \"kubernetes.io/projected/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-kube-api-access-6nqpc\") pod \"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc\" (UID: \"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc\") " Jul 31 19:35:24.690525 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:24.690497 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc" (UID: "165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 19:35:24.690644 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:24.690522 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-kube-api-access-6nqpc" (OuterVolumeSpecName: "kube-api-access-6nqpc") pod "165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc" (UID: "165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc"). InnerVolumeSpecName "kube-api-access-6nqpc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 19:35:24.789636 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:24.789548 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-webhook-cert\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:35:24.789636 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:24.789584 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nqpc\" (UniqueName: \"kubernetes.io/projected/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc-kube-api-access-6nqpc\") on node \"ip-10-0-136-28.ec2.internal\" DevicePath \"\"" Jul 31 19:35:25.293602 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:25.293573 2565 generic.go:356] "Generic (PLEG): container finished" podID="165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc" containerID="4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26" exitCode=0 Jul 31 19:35:25.293978 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:25.293613 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" Jul 31 19:35:25.293978 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:25.293654 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" event={"ID":"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc","Type":"ContainerDied","Data":"4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26"} Jul 31 19:35:25.293978 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:25.293709 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f6db4dfbf-nlwhk" event={"ID":"165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc","Type":"ContainerDied","Data":"1d4a2c24fe216464443c6092f77f64f7188f95342d6c30e00e5ac34e74d69f80"} Jul 31 19:35:25.293978 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:25.293737 2565 scope.go:117] "RemoveContainer" containerID="4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26" Jul 31 19:35:25.303897 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:25.303834 2565 scope.go:117] "RemoveContainer" containerID="4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26" Jul 31 19:35:25.304186 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:35:25.304158 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26\": container with ID starting with 4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26 not found: ID does not exist" containerID="4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26" Jul 31 19:35:25.304279 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:25.304196 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26"} err="failed to get container status \"4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26\": rpc error: code = NotFound desc = could not find container \"4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26\": container with ID starting with 4c82e25e6dab8f40ed66afa7752c51d3ea8cf492d88222736a79023a29ffbe26 not found: ID does not exist" Jul 31 19:35:25.309673 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:25.309645 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f6db4dfbf-nlwhk"] Jul 31 19:35:25.316224 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:25.316201 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f6db4dfbf-nlwhk"] Jul 31 19:35:27.264693 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:27.264662 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc" path="/var/lib/kubelet/pods/165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc/volumes" Jul 31 19:35:46.217547 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.217501 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2"] Jul 31 19:35:46.218006 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.217931 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc" containerName="manager" Jul 31 19:35:46.218006 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.217942 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc" containerName="manager" Jul 31 19:35:46.218079 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.218013 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="165f3dc3-a390-4b6f-a8b0-5a6bdb213bdc" containerName="manager" Jul 31 19:35:46.226504 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.226479 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:46.227926 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.227894 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2"] Jul 31 19:35:46.228496 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.228476 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 31 19:35:46.289417 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.289386 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6789dc-d639-4a63-a278-5b63f64702b2-openshift-service-ca\") pod \"maas-api-6695c997f5-mz8c2\" (UID: \"fc6789dc-d639-4a63-a278-5b63f64702b2\") " pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:46.289606 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.289433 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ccr8\" (UniqueName: \"kubernetes.io/projected/fc6789dc-d639-4a63-a278-5b63f64702b2-kube-api-access-6ccr8\") pod \"maas-api-6695c997f5-mz8c2\" (UID: \"fc6789dc-d639-4a63-a278-5b63f64702b2\") " pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:46.289606 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.289498 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc6789dc-d639-4a63-a278-5b63f64702b2-maas-api-tls\") pod \"maas-api-6695c997f5-mz8c2\" (UID: \"fc6789dc-d639-4a63-a278-5b63f64702b2\") " pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:46.390504 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.390465 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6789dc-d639-4a63-a278-5b63f64702b2-openshift-service-ca\") pod \"maas-api-6695c997f5-mz8c2\" (UID: \"fc6789dc-d639-4a63-a278-5b63f64702b2\") " pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:46.390504 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.390506 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccr8\" (UniqueName: \"kubernetes.io/projected/fc6789dc-d639-4a63-a278-5b63f64702b2-kube-api-access-6ccr8\") pod \"maas-api-6695c997f5-mz8c2\" (UID: \"fc6789dc-d639-4a63-a278-5b63f64702b2\") " pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:46.390770 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.390546 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc6789dc-d639-4a63-a278-5b63f64702b2-maas-api-tls\") pod \"maas-api-6695c997f5-mz8c2\" (UID: \"fc6789dc-d639-4a63-a278-5b63f64702b2\") " pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:46.390770 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:35:46.390697 2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 31 19:35:46.390869 ip-10-0-136-28 kubenswrapper[2565]: E0731 19:35:46.390780 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc6789dc-d639-4a63-a278-5b63f64702b2-maas-api-tls podName:fc6789dc-d639-4a63-a278-5b63f64702b2 nodeName:}" failed. No retries permitted until 2026-07-31 19:35:46.890761259 +0000 UTC m=+742.162108597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fc6789dc-d639-4a63-a278-5b63f64702b2-maas-api-tls") pod "maas-api-6695c997f5-mz8c2" (UID: "fc6789dc-d639-4a63-a278-5b63f64702b2") : secret "maas-api-serving-cert" not found Jul 31 19:35:46.391137 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.391120 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6789dc-d639-4a63-a278-5b63f64702b2-openshift-service-ca\") pod \"maas-api-6695c997f5-mz8c2\" (UID: \"fc6789dc-d639-4a63-a278-5b63f64702b2\") " pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:46.400828 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.400799 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ccr8\" (UniqueName: \"kubernetes.io/projected/fc6789dc-d639-4a63-a278-5b63f64702b2-kube-api-access-6ccr8\") pod \"maas-api-6695c997f5-mz8c2\" (UID: \"fc6789dc-d639-4a63-a278-5b63f64702b2\") " pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:46.895941 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.895907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc6789dc-d639-4a63-a278-5b63f64702b2-maas-api-tls\") pod \"maas-api-6695c997f5-mz8c2\" (UID: \"fc6789dc-d639-4a63-a278-5b63f64702b2\") " pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:46.898543 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:46.898516 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc6789dc-d639-4a63-a278-5b63f64702b2-maas-api-tls\") pod \"maas-api-6695c997f5-mz8c2\" (UID: \"fc6789dc-d639-4a63-a278-5b63f64702b2\") " pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:47.140528 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:47.140492 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:47.290016 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:47.289872 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2"] Jul 31 19:35:47.292434 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:35:47.292404 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc6789dc_d639_4a63_a278_5b63f64702b2.slice/crio-c78a0f28144925bef6212bb33a01f42154f4cd58081c841c317762455916ff76 WatchSource:0}: Error finding container c78a0f28144925bef6212bb33a01f42154f4cd58081c841c317762455916ff76: Status 404 returned error can't find the container with id c78a0f28144925bef6212bb33a01f42154f4cd58081c841c317762455916ff76 Jul 31 19:35:47.293982 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:47.293963 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 19:35:47.413771 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:47.413690 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" event={"ID":"fc6789dc-d639-4a63-a278-5b63f64702b2","Type":"ContainerStarted","Data":"c78a0f28144925bef6212bb33a01f42154f4cd58081c841c317762455916ff76"} Jul 31 19:35:49.425442 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:49.425359 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" event={"ID":"fc6789dc-d639-4a63-a278-5b63f64702b2","Type":"ContainerStarted","Data":"b233407a31a2463f3fc3407c5f15dbf6528cd676980ebcf3ab867c7a72dd2004"} Jul 31 19:35:49.425785 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:49.425479 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:49.459047 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:49.458994 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" podStartSLOduration=1.601362005 podStartE2EDuration="3.458976823s" totalImagesPullingTime="1.857614818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:35:49.457219171 +0000 UTC m=+744.728566525" watchObservedRunningTime="2026-07-31 19:35:49.458976823 +0000 UTC m=+744.730324180" Jul 31 19:35:55.436695 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:55.436661 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6695c997f5-mz8c2" Jul 31 19:35:56.000517 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:56.000472 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xnzmm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 31 19:35:56.000687 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:56.000554 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" podUID="d1b041eb-d946-4c32-bb75-490e0f781091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 31 19:35:56.465158 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:56.465126 2565 generic.go:356] "Generic (PLEG): container finished" podID="d1b041eb-d946-4c32-bb75-490e0f781091" containerID="b85d0ce1157f4c573b9b06d40f518d1b3bd3397cefde1cf26694a0e152d77591" exitCode=0 Jul 31 19:35:56.465582 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:56.465177 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" event={"ID":"d1b041eb-d946-4c32-bb75-490e0f781091","Type":"ContainerDied","Data":"b85d0ce1157f4c573b9b06d40f518d1b3bd3397cefde1cf26694a0e152d77591"} Jul 31 19:35:56.465626 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:56.465605 2565 scope.go:117] "RemoveContainer" containerID="b85d0ce1157f4c573b9b06d40f518d1b3bd3397cefde1cf26694a0e152d77591" Jul 31 19:35:56.466722 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:56.466686 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 31 19:35:56.466832 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:56.466771 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 31 19:35:56.466832 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:56.466820 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 31 19:35:57.473012 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:57.472981 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" event={"ID":"d1b041eb-d946-4c32-bb75-490e0f781091","Type":"ContainerStarted","Data":"e18ca98529b649ced7642beb3ed5e5041de07ca9e60c4dd0d290edb420a60c77"} Jul 31 19:35:57.989777 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:57.989742 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:35:57.990160 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:57.990064 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xnzmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 31 19:35:57.990328 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:57.990307 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" podUID="d1b041eb-d946-4c32-bb75-490e0f781091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 31 19:35:58.989848 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:58.989810 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xnzmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 31 19:35:58.990438 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:58.990407 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" podUID="d1b041eb-d946-4c32-bb75-490e0f781091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 31 19:35:59.990118 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:59.990078 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xnzmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 31 19:35:59.990706 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:35:59.990675 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" podUID="d1b041eb-d946-4c32-bb75-490e0f781091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 31 19:36:00.989608 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:00.989567 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xnzmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 31 19:36:00.989829 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:00.989639 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" podUID="d1b041eb-d946-4c32-bb75-490e0f781091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 31 19:36:01.989609 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:01.989570 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xnzmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 31 19:36:01.990603 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:01.990154 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" podUID="d1b041eb-d946-4c32-bb75-490e0f781091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 31 19:36:02.993090 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:02.993059 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:36:02.993501 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:02.993438 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:36:02.994039 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:02.994024 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xnzmm" Jul 31 19:36:08.357942 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.357909 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl"] Jul 31 19:36:08.360648 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.360628 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.362844 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.362823 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 31 19:36:08.363315 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.363295 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 31 19:36:08.363438 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.363322 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zvfxh\"" Jul 31 19:36:08.363438 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.363363 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 31 19:36:08.371192 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.371173 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl"] Jul 31 19:36:08.508423 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.508392 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.508587 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.508444 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.508587 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.508512 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.508587 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.508566 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtg69\" (UniqueName: \"kubernetes.io/projected/ba8aa247-6f83-4231-8685-60b343829122-kube-api-access-qtg69\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.508695 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.508604 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba8aa247-6f83-4231-8685-60b343829122-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.508695 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.508626 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.609579 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.609494 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.609579 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.609577 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.609808 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.609635 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.609808 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.609676 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.609808 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.609717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtg69\" (UniqueName: \"kubernetes.io/projected/ba8aa247-6f83-4231-8685-60b343829122-kube-api-access-qtg69\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.609808 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.609757 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba8aa247-6f83-4231-8685-60b343829122-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.610078 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.610042 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.610191 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.610113 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.610286 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.610269 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.612167 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.612146 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba8aa247-6f83-4231-8685-60b343829122-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.612316 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.612299 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba8aa247-6f83-4231-8685-60b343829122-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.618310 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.618287 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtg69\" (UniqueName: \"kubernetes.io/projected/ba8aa247-6f83-4231-8685-60b343829122-kube-api-access-qtg69\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl\" (UID: \"ba8aa247-6f83-4231-8685-60b343829122\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.672168 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.672130 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:08.809486 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:08.809460 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl"] Jul 31 19:36:08.811541 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:36:08.811514 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba8aa247_6f83_4231_8685_60b343829122.slice/crio-000dd6d508b68816ad842c5b86aead8929d05f6b3bf5a843c3cc5b6d2f1a45f8 WatchSource:0}: Error finding container 000dd6d508b68816ad842c5b86aead8929d05f6b3bf5a843c3cc5b6d2f1a45f8: Status 404 returned error can't find the container with id 000dd6d508b68816ad842c5b86aead8929d05f6b3bf5a843c3cc5b6d2f1a45f8 Jul 31 19:36:09.538306 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:09.538268 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" event={"ID":"ba8aa247-6f83-4231-8685-60b343829122","Type":"ContainerStarted","Data":"000dd6d508b68816ad842c5b86aead8929d05f6b3bf5a843c3cc5b6d2f1a45f8"} Jul 31 19:36:10.544877 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:10.544840 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" event={"ID":"ba8aa247-6f83-4231-8685-60b343829122","Type":"ContainerStarted","Data":"e26cee8083e613626aaf80e6c5899ffc353bcfc1a392eb001da0e1aaea6ec459"} Jul 31 19:36:10.562144 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:10.562102 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" podStartSLOduration=1.606349554 podStartE2EDuration="2.562086327s" totalImagesPullingTime="955.736773ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:36:10.560920876 +0000 UTC m=+765.832268232" watchObservedRunningTime="2026-07-31 19:36:10.562086327 +0000 UTC m=+765.833433683" Jul 31 19:36:18.672868 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:18.672823 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:18.673410 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:18.672903 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:18.686153 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:18.686126 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:36:19.601868 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:36:19.601839 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl" Jul 31 19:37:15.251186 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.251104 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc"] Jul 31 19:37:15.255030 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.254999 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.257206 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.257180 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 31 19:37:15.276702 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.276667 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc"] Jul 31 19:37:15.345230 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.345195 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m66s9\" (UniqueName: \"kubernetes.io/projected/1265f5d9-6430-41c9-ac2b-51d9c8806825-kube-api-access-m66s9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.345439 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.345289 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.345439 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.345382 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.345439 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.345409 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.345570 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.345456 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.345570 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.345509 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1265f5d9-6430-41c9-ac2b-51d9c8806825-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.446524 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.446482 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m66s9\" (UniqueName: \"kubernetes.io/projected/1265f5d9-6430-41c9-ac2b-51d9c8806825-kube-api-access-m66s9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.446700 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.446540 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.446700 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.446581 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.446700 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.446599 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.446700 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.446647 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.446700 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.446677 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1265f5d9-6430-41c9-ac2b-51d9c8806825-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.447092 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.447060 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.447092 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.447076 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.447092 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.447091 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.449152 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.449127 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1265f5d9-6430-41c9-ac2b-51d9c8806825-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.449433 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.449417 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1265f5d9-6430-41c9-ac2b-51d9c8806825-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.456507 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.456482 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m66s9\" (UniqueName: \"kubernetes.io/projected/1265f5d9-6430-41c9-ac2b-51d9c8806825-kube-api-access-m66s9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc\" (UID: \"1265f5d9-6430-41c9-ac2b-51d9c8806825\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.571181 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.571142 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:15.718297 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.718272 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc"] Jul 31 19:37:15.724643 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:37:15.724609 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1265f5d9_6430_41c9_ac2b_51d9c8806825.slice/crio-34245c388005aead07ca719c2cb79afc495bbc862a7c61d9f2ddd1e984753901 WatchSource:0}: Error finding container 34245c388005aead07ca719c2cb79afc495bbc862a7c61d9f2ddd1e984753901: Status 404 returned error can't find the container with id 34245c388005aead07ca719c2cb79afc495bbc862a7c61d9f2ddd1e984753901 Jul 31 19:37:15.894798 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:15.894703 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" event={"ID":"1265f5d9-6430-41c9-ac2b-51d9c8806825","Type":"ContainerStarted","Data":"34245c388005aead07ca719c2cb79afc495bbc862a7c61d9f2ddd1e984753901"} Jul 31 19:37:16.902483 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:16.902438 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" event={"ID":"1265f5d9-6430-41c9-ac2b-51d9c8806825","Type":"ContainerStarted","Data":"d7957aea0b8140dc71295a356026c40811e9fc0b529dc5a3d74e24585517e6b5"} Jul 31 19:37:16.922786 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:16.922721 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" podStartSLOduration=1.713826283 podStartE2EDuration="1.922697838s" totalImagesPullingTime="208.871555ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:37:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:37:16.92024576 +0000 UTC m=+832.191593115" watchObservedRunningTime="2026-07-31 19:37:16.922697838 +0000 UTC m=+832.194045195" Jul 31 19:37:25.572131 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:25.572095 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:25.572621 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:25.572191 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:25.585219 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:25.585183 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:37:25.961957 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:37:25.961867 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc" Jul 31 19:39:41.522448 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:41.522409 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-nshht_1b2f107d-07cd-4d05-a7f3-c40d3c30162b/manager/0.log" Jul 31 19:39:41.685785 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:41.685736 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jcw7n_c3ee0c60-5559-42b2-93a6-069acf139e49/manager/0.log" Jul 31 19:39:41.837829 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:41.837786 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bcf5f6d66-cvcwn_d2948fe6-b13c-4826-86d2-40858e0f8952/manager/0.log" Jul 31 19:39:41.967972 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:41.967940 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-krdgh_75366a00-6e93-450b-bb3f-54bd8dd9e195/server/0.log" Jul 31 19:39:42.097008 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:42.096934 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kqxl5_9830454e-4c7a-4be4-ade8-5ee6caabae0f/manager/0.log" Jul 31 19:39:42.206502 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:42.206473 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bbd966b49-8l77k_43c17712-de70-4e6f-81ad-45a33d71cefe/copy-manifests/0.log" Jul 31 19:39:42.223992 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:42.223963 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bbd966b49-8l77k_43c17712-de70-4e6f-81ad-45a33d71cefe/manager/0.log" Jul 31 19:39:42.786112 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:42.786066 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6695c997f5-mz8c2_fc6789dc-d639-4a63-a278-5b63f64702b2/maas-api/0.log" Jul 31 19:39:42.890717 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:42.890686 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-hz7vs_fe8480f6-5a0f-4539-ad3d-9644b5b3607d/postgres/0.log" Jul 31 19:39:43.746196 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:43.746158 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-6lmnx_d76b395e-3f70-44c1-b8b0-8077db201543/manager/0.log" Jul 31 19:39:43.854740 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:43.854692 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-k2b7d_fd0713f0-ddbe-4ebd-8aeb-34d636cee19f/manager/0.log" Jul 31 19:39:43.954514 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:43.954484 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-td4z7_bb36e754-d9e8-4528-8e48-e5bf33d8ceee/kuadrant-console-plugin/0.log" Jul 31 19:39:44.326512 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:44.326476 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-lvlmp_46f004da-30a5-4764-a582-8a6ddaa7bec6/manager/0.log" Jul 31 19:39:44.650200 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:44.650121 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6_5923e4f8-49e1-4a4e-91ac-0cc9023eb723/istio-proxy/0.log" Jul 31 19:39:44.999601 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:44.999519 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcff68c6d-bspch_7957d167-131d-46ca-b30f-9731ebb19684/kube-auth-proxy/0.log" Jul 31 19:39:45.104865 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:45.104828 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xnzmm_d1b041eb-d946-4c32-bb75-490e0f781091/istio-proxy/1.log" Jul 31 19:39:45.211478 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:45.211450 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56fb76c566-qls6z_ff4fac53-98d8-4c8f-b80c-6aec424edf2b/router/0.log" Jul 31 19:39:45.531521 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:45.531492 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-8z5cl_ba8aa247-6f83-4231-8685-60b343829122/main/0.log" Jul 31 19:39:45.852632 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:45.852560 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft4zbc_1265f5d9-6430-41c9-ac2b-51d9c8806825/main/0.log" Jul 31 19:39:51.727080 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.727044 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6j52w/must-gather-wq8n9"] Jul 31 19:39:51.730974 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.730952 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6j52w/must-gather-wq8n9" Jul 31 19:39:51.733061 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.733034 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6j52w\"/\"openshift-service-ca.crt\"" Jul 31 19:39:51.733061 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.733050 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6j52w\"/\"kube-root-ca.crt\"" Jul 31 19:39:51.733477 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.733464 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6j52w\"/\"default-dockercfg-8z97r\"" Jul 31 19:39:51.737813 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.737791 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6j52w/must-gather-wq8n9"] Jul 31 19:39:51.830920 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.830885 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0fc760c6-4a38-4fea-b6a5-fcb7fce0324b-must-gather-output\") pod \"must-gather-wq8n9\" (UID: \"0fc760c6-4a38-4fea-b6a5-fcb7fce0324b\") " pod="openshift-must-gather-6j52w/must-gather-wq8n9" Jul 31 19:39:51.831135 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.830930 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbncb\" (UniqueName: \"kubernetes.io/projected/0fc760c6-4a38-4fea-b6a5-fcb7fce0324b-kube-api-access-xbncb\") pod \"must-gather-wq8n9\" (UID: \"0fc760c6-4a38-4fea-b6a5-fcb7fce0324b\") " pod="openshift-must-gather-6j52w/must-gather-wq8n9" Jul 31 19:39:51.932142 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.932105 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0fc760c6-4a38-4fea-b6a5-fcb7fce0324b-must-gather-output\") pod \"must-gather-wq8n9\" (UID: \"0fc760c6-4a38-4fea-b6a5-fcb7fce0324b\") " pod="openshift-must-gather-6j52w/must-gather-wq8n9" Jul 31 19:39:51.932327 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.932149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbncb\" (UniqueName: \"kubernetes.io/projected/0fc760c6-4a38-4fea-b6a5-fcb7fce0324b-kube-api-access-xbncb\") pod \"must-gather-wq8n9\" (UID: \"0fc760c6-4a38-4fea-b6a5-fcb7fce0324b\") " pod="openshift-must-gather-6j52w/must-gather-wq8n9" Jul 31 19:39:51.932493 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.932474 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0fc760c6-4a38-4fea-b6a5-fcb7fce0324b-must-gather-output\") pod \"must-gather-wq8n9\" (UID: \"0fc760c6-4a38-4fea-b6a5-fcb7fce0324b\") " pod="openshift-must-gather-6j52w/must-gather-wq8n9" Jul 31 19:39:51.939843 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:51.939822 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbncb\" (UniqueName: \"kubernetes.io/projected/0fc760c6-4a38-4fea-b6a5-fcb7fce0324b-kube-api-access-xbncb\") pod \"must-gather-wq8n9\" (UID: \"0fc760c6-4a38-4fea-b6a5-fcb7fce0324b\") " pod="openshift-must-gather-6j52w/must-gather-wq8n9" Jul 31 19:39:52.041116 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:52.041024 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6j52w/must-gather-wq8n9" Jul 31 19:39:52.182107 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:52.179982 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6j52w/must-gather-wq8n9"] Jul 31 19:39:52.184023 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:39:52.183993 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fc760c6_4a38_4fea_b6a5_fcb7fce0324b.slice/crio-c7e5625918d4dde8e05f88eebfadb7b2e834b53648207d436942ce1a43b2a4d8 WatchSource:0}: Error finding container c7e5625918d4dde8e05f88eebfadb7b2e834b53648207d436942ce1a43b2a4d8: Status 404 returned error can't find the container with id c7e5625918d4dde8e05f88eebfadb7b2e834b53648207d436942ce1a43b2a4d8 Jul 31 19:39:52.704784 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:52.704752 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6j52w/must-gather-wq8n9" event={"ID":"0fc760c6-4a38-4fea-b6a5-fcb7fce0324b","Type":"ContainerStarted","Data":"c7e5625918d4dde8e05f88eebfadb7b2e834b53648207d436942ce1a43b2a4d8"} Jul 31 19:39:53.712419 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:53.712377 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6j52w/must-gather-wq8n9" event={"ID":"0fc760c6-4a38-4fea-b6a5-fcb7fce0324b","Type":"ContainerStarted","Data":"bcb36741ab422bb616bf88b616501118610079c8ab9a1d578edf6d01aa34ee6c"} Jul 31 19:39:53.712883 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:53.712434 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6j52w/must-gather-wq8n9" event={"ID":"0fc760c6-4a38-4fea-b6a5-fcb7fce0324b","Type":"ContainerStarted","Data":"e5e2de42a0ef47ea3ecae926812f276638907d50ef6b8902f320ed84a2133399"} Jul 31 19:39:53.728462 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:53.728396 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6j52w/must-gather-wq8n9" podStartSLOduration=2.072365806 podStartE2EDuration="2.728372985s" totalImagesPullingTime="656.007179ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 19:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:39:53.725612266 +0000 UTC m=+988.996959621" watchObservedRunningTime="2026-07-31 19:39:53.728372985 +0000 UTC m=+988.999720339" Jul 31 19:39:54.501516 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:54.501484 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xn2mh_0efd55cb-5725-4a4c-9e3b-3725b3041fdc/global-pull-secret-syncer/0.log" Jul 31 19:39:54.577463 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:54.577430 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hpnqm_a74545a8-857f-4388-9bd0-4b015d5acaf7/konnectivity-agent/0.log" Jul 31 19:39:54.666358 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:54.666308 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-28.ec2.internal_8d9b7ef00ef8d5b576fd1871ffcc6c95/haproxy/0.log" Jul 31 19:39:59.184164 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:59.184131 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-6lmnx_d76b395e-3f70-44c1-b8b0-8077db201543/manager/0.log" Jul 31 19:39:59.215290 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:59.215248 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-k2b7d_fd0713f0-ddbe-4ebd-8aeb-34d636cee19f/manager/0.log" Jul 31 19:39:59.241122 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:59.241087 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-td4z7_bb36e754-d9e8-4528-8e48-e5bf33d8ceee/kuadrant-console-plugin/0.log" Jul 31 19:39:59.376536 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:39:59.376507 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-lvlmp_46f004da-30a5-4764-a582-8a6ddaa7bec6/manager/0.log" Jul 31 19:40:00.906888 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:00.906853 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57f53419-1aeb-4cae-8912-604df35afc71/alertmanager/0.log" Jul 31 19:40:00.929791 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:00.929755 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57f53419-1aeb-4cae-8912-604df35afc71/config-reloader/0.log" Jul 31 19:40:00.954996 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:00.954935 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57f53419-1aeb-4cae-8912-604df35afc71/kube-rbac-proxy-web/0.log" Jul 31 19:40:00.977626 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:00.977596 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57f53419-1aeb-4cae-8912-604df35afc71/kube-rbac-proxy/0.log" Jul 31 19:40:01.002965 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.002928 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57f53419-1aeb-4cae-8912-604df35afc71/kube-rbac-proxy-metric/0.log" Jul 31 19:40:01.028329 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.028298 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57f53419-1aeb-4cae-8912-604df35afc71/prom-label-proxy/0.log" Jul 31 19:40:01.066590 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.066559 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_57f53419-1aeb-4cae-8912-604df35afc71/init-config-reloader/0.log" Jul 31 19:40:01.101380 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.101316 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-98vlx_3207a5bd-e828-4d3b-9687-62e26d3b5f3b/cluster-monitoring-operator/0.log" Jul 31 19:40:01.138926 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.138893 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-zfmgz_e2fb96e5-3889-4143-85a5-b9dc5d1d6cee/kube-state-metrics/0.log" Jul 31 19:40:01.162812 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.162734 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-zfmgz_e2fb96e5-3889-4143-85a5-b9dc5d1d6cee/kube-rbac-proxy-main/0.log" Jul 31 19:40:01.192399 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.192366 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-zfmgz_e2fb96e5-3889-4143-85a5-b9dc5d1d6cee/kube-rbac-proxy-self/0.log" Jul 31 19:40:01.244145 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.244110 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-pz4xj_4d7df320-71f8-4fc2-9938-dc27f5e609f1/monitoring-plugin/0.log" Jul 31 19:40:01.441856 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.441771 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrz6v_97b206c3-b784-4ea5-a74a-cd75d79ba7ad/node-exporter/0.log" Jul 31 19:40:01.464612 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.464585 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrz6v_97b206c3-b784-4ea5-a74a-cd75d79ba7ad/kube-rbac-proxy/0.log" Jul 31 19:40:01.487555 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.487532 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrz6v_97b206c3-b784-4ea5-a74a-cd75d79ba7ad/init-textfile/0.log" Jul 31 19:40:01.992896 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:01.992862 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/thanos-query/0.log" Jul 31 19:40:02.022891 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:02.022815 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/kube-rbac-proxy-web/0.log" Jul 31 19:40:02.047770 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:02.047719 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/kube-rbac-proxy/0.log" Jul 31 19:40:02.068833 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:02.068799 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/prom-label-proxy/0.log" Jul 31 19:40:02.092884 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:02.092852 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/kube-rbac-proxy-rules/0.log" Jul 31 19:40:02.113735 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:02.113704 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f6db489c-bptgv_e1aca7fa-d134-434e-9104-827c90c78f8b/kube-rbac-proxy-metrics/0.log" Jul 31 19:40:03.583855 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:03.583821 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-b5t72_e977c3b5-b744-453b-9138-8f37b2facb8b/networking-console-plugin/0.log" Jul 31 19:40:04.490293 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.490257 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879"] Jul 31 19:40:04.496142 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.496111 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.499495 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.499382 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879"] Jul 31 19:40:04.573365 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.572817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-proc\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.573365 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.572868 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-podres\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.573365 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.572923 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-sys\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.573365 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.573003 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-lib-modules\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.573365 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.573074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-828jn\" (UniqueName: \"kubernetes.io/projected/20955d3a-845d-4557-8b0c-a6d6283685af-kube-api-access-828jn\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.674291 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.674254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-lib-modules\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.674772 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.674495 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-lib-modules\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.674772 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.674539 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-828jn\" (UniqueName: \"kubernetes.io/projected/20955d3a-845d-4557-8b0c-a6d6283685af-kube-api-access-828jn\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.674772 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.674607 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-proc\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.674772 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.674634 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-podres\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.674772 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.674683 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-sys\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.675028 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.674797 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-sys\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.675028 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.674854 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-proc\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.675028 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.674945 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/20955d3a-845d-4557-8b0c-a6d6283685af-podres\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.682435 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.682398 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-828jn\" (UniqueName: \"kubernetes.io/projected/20955d3a-845d-4557-8b0c-a6d6283685af-kube-api-access-828jn\") pod \"perf-node-gather-daemonset-sv879\" (UID: \"20955d3a-845d-4557-8b0c-a6d6283685af\") " pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:04.809852 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:04.809767 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:05.015556 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:05.015526 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879"] Jul 31 19:40:05.016853 ip-10-0-136-28 kubenswrapper[2565]: W0731 19:40:05.016826 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod20955d3a_845d_4557_8b0c_a6d6283685af.slice/crio-350b2c487d79e92009f7b4f071af92585956f9d573d8f0b489441cc5456b119c WatchSource:0}: Error finding container 350b2c487d79e92009f7b4f071af92585956f9d573d8f0b489441cc5456b119c: Status 404 returned error can't find the container with id 350b2c487d79e92009f7b4f071af92585956f9d573d8f0b489441cc5456b119c Jul 31 19:40:05.496590 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:05.496559 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-drtx9_649255b5-2ff9-4521-a5c8-01b6cde60ea9/volume-data-source-validator/0.log" Jul 31 19:40:05.810578 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:05.810483 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" event={"ID":"20955d3a-845d-4557-8b0c-a6d6283685af","Type":"ContainerStarted","Data":"72ddb8357288dcd3f34c174112d4e48ecabcdd4a946c458087f45a096ab15858"} Jul 31 19:40:05.811143 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:05.811114 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" event={"ID":"20955d3a-845d-4557-8b0c-a6d6283685af","Type":"ContainerStarted","Data":"350b2c487d79e92009f7b4f071af92585956f9d573d8f0b489441cc5456b119c"} Jul 31 19:40:05.811292 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:05.811280 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:05.826160 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:05.826107 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" podStartSLOduration=1.826091785 podStartE2EDuration="1.826091785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 19:40:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 19:40:05.825003255 +0000 UTC m=+1001.096350610" watchObservedRunningTime="2026-07-31 19:40:05.826091785 +0000 UTC m=+1001.097439169" Jul 31 19:40:06.571998 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:06.571960 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lk42m_4dd7e20b-fce0-4861-a2b8-39e6e751dfb1/dns/0.log" Jul 31 19:40:06.592241 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:06.592214 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lk42m_4dd7e20b-fce0-4861-a2b8-39e6e751dfb1/kube-rbac-proxy/0.log" Jul 31 19:40:06.679426 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:06.679397 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mkm94_9495893a-b14e-4fd9-8999-226d2b5d4eea/dns-node-resolver/0.log" Jul 31 19:40:07.241390 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:07.241360 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5lpws_5506188d-51d5-4bfa-b204-a8e580d3f1b2/node-ca/0.log" Jul 31 19:40:08.189629 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:08.189597 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6_5923e4f8-49e1-4a4e-91ac-0cc9023eb723/istio-proxy/0.log" Jul 31 19:40:08.290328 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:08.290296 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcff68c6d-bspch_7957d167-131d-46ca-b30f-9731ebb19684/kube-auth-proxy/0.log" Jul 31 19:40:08.319616 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:08.319582 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xnzmm_d1b041eb-d946-4c32-bb75-490e0f781091/istio-proxy/0.log" Jul 31 19:40:08.320205 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:08.320182 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xnzmm_d1b041eb-d946-4c32-bb75-490e0f781091/istio-proxy/1.log" Jul 31 19:40:08.345487 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:08.345458 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56fb76c566-qls6z_ff4fac53-98d8-4c8f-b80c-6aec424edf2b/router/0.log" Jul 31 19:40:08.972147 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:08.972121 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tdqjh_9fcd2bdb-ce6d-4218-93c6-e40901c0dd2d/serve-healthcheck-canary/0.log" Jul 31 19:40:09.457204 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:09.457174 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5552f_9be97ade-59bb-4c0c-8c59-3a81ab658e8d/kube-rbac-proxy/0.log" Jul 31 19:40:09.478789 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:09.478750 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5552f_9be97ade-59bb-4c0c-8c59-3a81ab658e8d/exporter/0.log" Jul 31 19:40:09.499816 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:09.499782 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5552f_9be97ade-59bb-4c0c-8c59-3a81ab658e8d/extractor/0.log" Jul 31 19:40:11.671058 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:11.671018 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-nshht_1b2f107d-07cd-4d05-a7f3-c40d3c30162b/manager/0.log" Jul 31 19:40:11.709165 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:11.709135 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jcw7n_c3ee0c60-5559-42b2-93a6-069acf139e49/manager/0.log" Jul 31 19:40:11.744008 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:11.743977 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bcf5f6d66-cvcwn_d2948fe6-b13c-4826-86d2-40858e0f8952/manager/0.log" Jul 31 19:40:11.761840 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:11.761812 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-krdgh_75366a00-6e93-450b-bb3f-54bd8dd9e195/server/0.log" Jul 31 19:40:11.831110 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:11.831080 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kqxl5_9830454e-4c7a-4be4-ade8-5ee6caabae0f/manager/0.log" Jul 31 19:40:11.893875 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:11.893835 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bbd966b49-8l77k_43c17712-de70-4e6f-81ad-45a33d71cefe/manager/0.log" Jul 31 19:40:11.912363 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:11.912326 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bbd966b49-8l77k_43c17712-de70-4e6f-81ad-45a33d71cefe/copy-manifests/0.log" Jul 31 19:40:12.838966 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:12.838935 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6j52w/perf-node-gather-daemonset-sv879" Jul 31 19:40:14.680285 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:14.680248 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6_5923e4f8-49e1-4a4e-91ac-0cc9023eb723/istio-proxy/0.log" Jul 31 19:40:14.766774 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:14.766740 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcff68c6d-bspch_7957d167-131d-46ca-b30f-9731ebb19684/kube-auth-proxy/0.log" Jul 31 19:40:14.797290 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:14.797209 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xnzmm_d1b041eb-d946-4c32-bb75-490e0f781091/istio-proxy/0.log" Jul 31 19:40:14.797771 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:14.797692 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xnzmm_d1b041eb-d946-4c32-bb75-490e0f781091/istio-proxy/1.log" Jul 31 19:40:14.813697 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:14.813672 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56fb76c566-qls6z_ff4fac53-98d8-4c8f-b80c-6aec424edf2b/router/0.log" Jul 31 19:40:16.095141 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:16.095103 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-d7kb6_5923e4f8-49e1-4a4e-91ac-0cc9023eb723/istio-proxy/0.log" Jul 31 19:40:16.171544 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:16.171501 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcff68c6d-bspch_7957d167-131d-46ca-b30f-9731ebb19684/kube-auth-proxy/0.log" Jul 31 19:40:16.193992 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:16.193943 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xnzmm_d1b041eb-d946-4c32-bb75-490e0f781091/istio-proxy/0.log" Jul 31 19:40:16.194707 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:16.194677 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xnzmm_d1b041eb-d946-4c32-bb75-490e0f781091/istio-proxy/1.log" Jul 31 19:40:16.208425 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:16.208396 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56fb76c566-qls6z_ff4fac53-98d8-4c8f-b80c-6aec424edf2b/router/0.log" Jul 31 19:40:18.649962 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:18.649930 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-x8pbg_ce3db913-8572-4f5b-951f-b4e3ee74f5da/kube-storage-version-migrator-operator/1.log" Jul 31 19:40:18.651531 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:18.651507 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-x8pbg_ce3db913-8572-4f5b-951f-b4e3ee74f5da/kube-storage-version-migrator-operator/0.log" Jul 31 19:40:19.639024 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:19.638994 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9762k_c69f371c-2aba-473f-8df1-01b116efe170/kube-multus/0.log" Jul 31 19:40:19.661259 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:19.661231 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5pmbt_42cde026-7512-4487-a633-846e325b55cc/kube-multus-additional-cni-plugins/0.log" Jul 31 19:40:19.680789 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:19.680769 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5pmbt_42cde026-7512-4487-a633-846e325b55cc/egress-router-binary-copy/0.log" Jul 31 19:40:19.701914 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:19.701889 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5pmbt_42cde026-7512-4487-a633-846e325b55cc/cni-plugins/0.log" Jul 31 19:40:19.721296 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:19.721273 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5pmbt_42cde026-7512-4487-a633-846e325b55cc/bond-cni-plugin/0.log" Jul 31 19:40:19.740859 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:19.740838 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5pmbt_42cde026-7512-4487-a633-846e325b55cc/routeoverride-cni/0.log" Jul 31 19:40:19.760497 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:19.760477 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5pmbt_42cde026-7512-4487-a633-846e325b55cc/whereabouts-cni-bincopy/0.log" Jul 31 19:40:19.781131 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:19.781113 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5pmbt_42cde026-7512-4487-a633-846e325b55cc/whereabouts-cni/0.log" Jul 31 19:40:20.174166 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:20.174128 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9qvpw_b9991931-cf33-4a99-8185-b3ba855761bc/network-metrics-daemon/0.log" Jul 31 19:40:20.192466 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:20.192437 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9qvpw_b9991931-cf33-4a99-8185-b3ba855761bc/kube-rbac-proxy/0.log" Jul 31 19:40:21.018717 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:21.018677 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2bmz_0650e893-0096-4562-8d1e-aa06dc59efc0/ovn-controller/0.log" Jul 31 19:40:21.035764 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:21.035728 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2bmz_0650e893-0096-4562-8d1e-aa06dc59efc0/ovn-acl-logging/0.log" Jul 31 19:40:21.063239 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:21.063211 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2bmz_0650e893-0096-4562-8d1e-aa06dc59efc0/kube-rbac-proxy-node/0.log" Jul 31 19:40:21.087245 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:21.087221 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2bmz_0650e893-0096-4562-8d1e-aa06dc59efc0/kube-rbac-proxy-ovn-metrics/0.log" Jul 31 19:40:21.103239 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:21.103217 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2bmz_0650e893-0096-4562-8d1e-aa06dc59efc0/northd/0.log" Jul 31 19:40:21.121579 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:21.121558 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2bmz_0650e893-0096-4562-8d1e-aa06dc59efc0/nbdb/0.log" Jul 31 19:40:21.140561 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:21.140531 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2bmz_0650e893-0096-4562-8d1e-aa06dc59efc0/sbdb/0.log" Jul 31 19:40:21.195120 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:21.195090 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2bmz_0650e893-0096-4562-8d1e-aa06dc59efc0/ovnkube-controller/0.log" Jul 31 19:40:22.748378 ip-10-0-136-28 kubenswrapper[2565]: I0731 19:40:22.748324 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-7rq9h_8ba86165-95a0-400b-8168-06860a6ad13a/network-check-target-container/0.log"